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ABSTRACT 



The report summarizes the results of the National Evaluation Program 
(NEP), Phase I Study of Prosecution Management Information Systems (PMIS's). 

The report includes discussions and findings regarding: the nature and 

scope of PMIS use; state-of-the-art in PMIS use; transfer potential of PMIS 
projects; funding of PMIS systems; organization of PMIS projects; the avail- 
ability of evaluation data; and the feasibility of PMIS evaluations. 

Significant findings include: 

1. Integrated systems, supported by a centralized computer facility 
serving multiple criminal justice agencies that share the data base, are the 
most common and most effective type of PMIS. 

2. Existing automated systems are being used effectively, in general, 
to support prosecutors' day-to-day case processing operations. However, few 
prosecutors, with the exception of PROMIS users, are using their PMIS as a 
strategic management tool to support their policy development and decision- 
making processes. The periodic exchange of information and ideas among 
prosecutors attending PROMIS User Group meetings appears to be the reason for 
this exception. PMIS contributions to attainment of prosecutor goals have 
been less than expected by the prosecutors. This situation also indicates .a 
need for prosecutors to exchange information about PMIS operations to either • 
develop more realistic expectations or to learn about applications that will 
help their PMIS to meet their expectations. 

3. PMIS transfer has not yet proven less costly than development from 
scratch as a means of acquiring an automated system. In previous PMIS trans- 
fers, agencies have tended to underestimate the difficulty of modifying the 
incoming system to meet local requirements. Frequently, technical assistance 
should be sought from experienced data processors to effectively implement the 



4. Federal funding has been a primary factor in both the transfer and 
local development of PMIS projects. Local jurisdictions have, thus far, 
assumed the financial burden of supporting PMIS operations when Federal funds 
run out. Future PMIS implementations will continue to rely heavily on Federal 
funding and local jurisdictions will be expected to fund future PMIS operations 
when Federal funds are exhausted. Allocation of funds for PMIS development, 
transfer and operations should encourage the capture of benefits resulting 
from previous investments in such systems at all levels of government. 

5. Data for PMIS cost-benefit analysis and feasible methods for mea- 
suring PMIS impact on the criminal justice system are available to support 
in-depth PMIS evaluations. Independent PMIS evaluations are needed to provide 
Federal, state and local government officials the information that they need 
to justify PMIS funding. 
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EXECUTIVE SUMMARY 



This National Evaluation Program (NEP), Phase I Study of Prosecution 
Management Information Systems, sponsored by the Law Enforcement Assistance 
Administration (LEAA) , was conducted by Westat, Inc. in affiliation with the 
Center for Management and Policy Research, Incorporated (CMPR). 

For this study, a prosecution management information system (PMIS) is 
defined as the people, procedures, hardware (e.g. , office and computer equip- 
ment), and software (e.g., computer programs and associated documentation) 
used by a prosecutor to create and use cas'e information to achieve prosecution 
goals. 

The study was conducted in two segments. The objectives of the first 
segment were: to identify and describe the nature and extent of PMIS usage 

and the problems, costs and benefits of such use; and to determine the avail- 
ability of data for in-depth evaluations. Information to meet these objec- 
tives was gathered by a literature search, mail and telephone surveys of more 
than 600 prosecution agencies, on-site visits to 17 operational PMIS projects 
and discussions with members of the nationwide criminal justice community. 

The second segment of the study was designed to determine the extent to 
which PMIS’s can be objectively evaluated and to test evaluation methods. 

These objectives were met by performing evaluation feasibility tests using 
data collected from six PMIS projects that were selected as test sites. 

Increasing criminal caseloads combined with limited resources have moti- 
vated many prosecutors to consider automation as a means of improving office 
operations and the management of resources. Survey data, collected during the 
first segment of this study, indicate that PMIS projects are being used or 
planned extensively among larger prosecution offices. About 80 percent of 
offices with 25 employees or more either use a PMIS or are planning a PMIS. 

Of those reportedly using a PMIS, 37 percent had some version of PROMIS. Of , 

rep ° rting . plans 1:0 use a percent were planning some version of 

PROMIS, indicating a trend toward the use of PROMIS. (For the purpose of this 
report, the acronym "PROMIS" refers to a group of software packages developed 
for LEAA by the Institute for Law and Social Research (INSLAW); the acronym 
"PMIS" includes both PROMIS and non-PROMIS systems.) 

Based on survey data, it is estimated that more than $30 million has been • 
spent over the past 10 years on PMIS development and in excess of $20 million 
is spent each year on PMIS operations. PMIS evaluations will be useful to 
Federal officials and state planning agencies in assessing PMIS funding 
policies and allocating funds among competing projects, particularly in light 
of diminishing criminal justice budgets. Criminal justice agencies that use a 
PMIS and data processing facilities that operate them will find PMIS evalua- 
tions useful in budget negotiations because PMIS costs and benefits will be 
identified and PMIS impacts on the criminal justice system will be defined. 





Evaluation results will' also assist PMIS users and operators in determining 
changes or additions to PMIS operations to make them more effective. 

Several sets of issues were developed during the course of the study. 

The first set of issues concerns the organizational context of PMIS projects. 
Several types of PMIS projects were observed. One type is the prosecutor 
dedicated PMIS. In 'this type of system, a small computer is housed in the 
prosecutor's office complex and operated by members of the prosecutor's staff; 
only prosecution personnel have direct access to the system, although PMIS 
outputs (e.g., schedules and caseload reports) may be provided to other 
agencies, such as the courts. Another type of project is where the computer 
system is housed and operated by one agency (e.g., the prosecutor), and data 
is shared with another agency (e.g., the courts or the police) via direct, 
on-line access to the PMIS. For the purpose of this study, these systems are 
said to have "limited data sharing." 

The other types of projects observed are called, here, "integrated sys- 
tems." In these types of systems, a central data processing facility (city, 
county or state) operates a large scale computer in support of PMIS opera- 
tions, and the PMIS data base is shared by several criminal justice agencies. 
Among integrated systems, data may be shared within a jurisdiction by several 
criminal justice agencies (e.g., police, prosecutor, courts, and corrections); 
data may be shared between jurisdictions (e.g., on a regional basis); or data 
may be shared on a statewide basis. 

Integrated systems, as a group, are used by most of the jurisdictions 
surveyed: among respondents to the mail and telephone surveys, 80 percent 

indicated that their PMIS is shared extensively; of the projects included in 
the site surveys, 82 percent operate integrated systems. It is difficult to 
secure the interagency cooperation needed to develop an effective integrated 
criminal justice information system involving extensive data sharing among 
criminal justice agencies. Yet, in jurisdictions where such extensive data 
sharing was observed, the projects appear to be more cost-effective than those 
projects characterized by limited data sharing. * 

The second set of issues concerns PMIS development and operation. Due to 
budgetary and staffing constraints, an essential ingredient of the wide adop- 
tion of PMIS's has been LEAA funding support. LEAA policies have encouraged 
technology transfer, as evidenced by the strong interest in PROMIS with 37 
jurisdictions operating some version of PROMIS and 134 other jurisdictions 
either transferring or planning PROMIS.- There is little evidence from this 
study to support the theory that PMIS transfer is less costly than development 
from scratch; in fact, respondents to the mail survey reported median develop- 
ment costs of $150,000 for 15 non-PROMIS projects and $175,000 for nine PROMIS 
projects. Several possible reasons for this are explored in the text. Exami- 
nation of the state-of-the-art in PMIS use focused on applications and the 
degree to which they are employed for . management purposes. The most common 
applications of PMIS projects were directed at day-to-day operations by 
improving scheduling and logistical control through case and defendant 
tracking and caseload reporting. The use of PMIS outputs to support strategic 
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management policy development and decisionmaking was not a' common practice, 
particularly in non-PROMIS sites where many prosecution managers do not 
understand the potential of the PMIS as a management tool. 

The third set of issues concerns the impact of PMIS projects on the 
criminal justice system. Prosecutors were able to rate the relative impor- 
tance of various PMIS related goals of their offices. They ranked "improve 
scheduling and logistical control" and "allocation of staff based on prose- 
cution priorities" most highly. Some operational PMIS projects have fallen 
short of prosecutors' expectations; however, this varies with different 
applications and types of systems. In general, survey results indicate that 
prosecution management's statistical information needs are satisfied more 
often by PRQMIS projects than by non-PROMIS projects, while the reverse holds 
for day-to-day operational support needs, such as case and defendant status 
reporting. A majority of the PMIS projects report that they generate measures 
of workload, court delay, and disposition rates. Such performance measures 
are useful in assessing PMIS impact. The collective experience in prosecution 
MIS development and operation have advanced to the point where uniform cri- 
teria can be applied and comparable measures of costs and performance can be 
developed for groups of projects. Such an in-depth analysis has been designed 
for a proposed Phase II NEP study. ' 

In response to a special evaluative interest expressed by LE.AA, the 
final set of issues concerns the evaluation of PROMIS projects as a group. 
Several versions of PROMIS were identified, some markedly different from 
others. Indeed even for particular versions, the study found that transfers 
often involved substantial software and procedural modifications., This may 
have limited the advantages of PROMIS transfer in lowering development costs 
and facilitating cross-city comparisons. Without the application of uniform 
criteria and measurement technigues, it is difficult to compare relative costs 
of projects. Based on the limited data available from the Phase I surveys, 
little evidence was found to support the notion that such transfers lowered 
development costs. A limited scope, cross-city analysis of PROMIS projects was 
accomplished by INSLAW. However, the effort involved was time' consuming and ‘ 
difficult despite the aid of a generalized report generator package, supplied 
by INSLAW. What appears to be special about the effort was the continuity of 
funding and cooperative working relationship established between the jursidic- 
tions and the contractor, rather than intrinsic attributes of the PROMIS 
data bases. The text of this report provides further evaluative information 
on- the various PROMIS versions. Present LEAA funding policies favor the use 
of PROMIS, and a new PROMIS software version, with attractive tailoring 
features, is being implemented. 

Also in response to LEAA concerns about how future PMIS funds should be 
allocated, the study team developed recommendations that funds be allocated on 
a priority basis: first, to provide technical assistance for PROMIS transfer, 

in order to realize maximum benefit from the heavy investment already made to 
this system; second, to jurisdictions seeking alternative approaches to 
PROMIS, because some alternatives may have more innovative and economical 
applications; and third, for the development of new PROMIS applications (as 



ix 





opposed to development of new systems), to provide even greater flexibilities 
to those PROMIS versions that already exist. Assigning priorities in this 
manner will help capture a return on investments already made in systems 
development and transfers by the Federal and local jurisdictions, and will • 
allow prosecutors to gain more experience with existing applications before 
investing in new enhancements. 

Data collection experiences and testing of evaluation methodologies, 
performed during the feasibility study, have resulted in the findings that: 
following a period of 24-26 months of PMIS operations, data are available from 
PMIS projects in sufficient detail to support cost-benefit analysis; PMIS 
performance measures can be derived from records such as the PMIS computer 
files, statistical reports generated either by the PMIS or manually, and/or 
abstracted from manually kept case files; and time series analysis, employing 
factor analysis techniques and supplemented by a set of control variables 
derived from on-site interviews and observations, provide feasible though not 
infallible methods for evaluating the PMIS impact on the criminal justice 
system. 

The field investigations reported here have been necessarily limited in 
scope. They have indicated the feasibility of conducting in-depth evaluations 
of PMIS projects, but. they have not constituted such in-depth evaluations. 
Also, with just six test sites, the sample is too small to do any cross- 
sectional analysis. The latter type of design, coupled with time series anal- 
ysis, offers the best opportunity for performing in-depth evaluations. Thus, 
it remains to examine the feasibility of constructing and analyzing a cross- 
jurisdictional time series data base for evaluating PROMIS and non-PROMIS 
projects. In the process of constructing and analyzing such a data base, a 
set of actual in-depth evaluations should be carried out as an extension of 
the research methodology tested in the field investigations reported here. 

The research should include the use of non-PMIS generated data to supplement 
the analysis. 

Several classes of PMIS projects were identified during the study. 
Projects classified as integrated systems with data sharing among criminal * 
justice agencies appear to be more cost effective than others. A cooperative 
research project involving about 30 jurisdictions in constructing' a cross- 
sectioned time series data base would create a powerful vehicle for performing 
in-depth evaluative research.* It would also create the opportunity for 
exchanging user information, disseminating research results and, in general, 
educating and upgrading PMIS users. By including both PROMIS and non-PROMIS 
users, more broadly applicable and realistic evaluations and exchanges of 
information would result. This would provide the greatest opportunity to 
assure that the PMIS is well understood and used for both operational and 
management purposes, and that the impact of such use can be measured and 
evaluated. The implications of such research for the Bureau of Justice 



♦Much of the necessary data are being compiled under the National Institute 
of Justice Cross-Jurisdictional Analysis. . 
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Statistics program are apparent. The participating PMIS projects could be 
used to generate a wide array of comparable criminal justice statistics in 
support of both longitudinal and cross-sectional analysis. 

Another issue to be explored in evaluating PMIS projects is the predic- 
tion of criminal justice system costs and performance. The application of 
stochastic (probabilistic) modeling and simulation techniques should be 
explored because such predictions are relevant in evaluating PMIS projects. 
Comparisons of actual with predicted results should assist in advancing the 
state of the art of evaluative and modeling research, as well as being intrin- 
sically useful in improving PMIS and criminal justice statistics programs. 

An evaluation of the comparative utility and of PMIS generated statistics 
versus periodic surveys by independent statistical agencies (Census or BLS 
models) should be performed. There are inherent advantages and disadvantages 
in developing and utilizing both approaches in an ongoing B3S sponsored 
statistical program. 
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A. Purpose of Report 

This report summarizes the results of the National Evaluation Program, 
Phase I Study of Prosecution Management Information Systems, conducted for 
£ the Law Enforcement Assistance Administration (LEAA) by Westat, Incorporated 

of Rockville, Maryland in affiliation with the Center for Management and 
Policy Research, Incorporated of Washington, D.C. It provides a description 
of the methodology used in the study and a summary of the issues and findings. 



■'1 

.1 





't: 



■>< 



£ 



E 



£ 



% 



I 



! « 



f 

! 



B. Background 

The prosecutor, in addition to prosecutorial duties, is responsible for 
management of the information that flows through his or her office. The 
information of primary concern to the prosecutor pertains to cases filed in 
court for prosecution. Case information includes data regarding: the de- 

fendant; the offense; witnesses;' investigations; assignment of assistant 
prosecutors, defense attorneys and judges; schedules for hearings, trials and 
sentencing; and, final disposition of the cases. 

The people, procedures, hardware, and software used by the prosecutor to 
manage case information is called, for the purpose of this study, a Prosecu- 
tion Management Information System (PMIS). 

A PMIS may be operated by manual methods; it may be automated by em- 
ploying a general purpose computer; or, it may be semi-automated, that is, it 
may function by using a combination of manual and automated techniques. 

In a manual system, the procedures include: the origination of data by 

preparing documents in handwritten form or by use of a typewriter; the distri- 
bution of those documents to users of the information; and, storage of those 
documents in containers such as file cabinets. The hardware, then, consists 
of the office equipment (typewriters and file cabinets, for example) used to 
prepare and store the data. The manual PMIS was excluded from the scope of 
of this study. 



An automated PMIS uses a computer to store, manipulate, retrieve, and 
disseminate case information. Computer capabilities provide prosecution 
personnel a quick and easy method of tracking cases and defendants; and they 
permit the automatic production of reports to support daily operations (e.g., 
schedules and dockets), management of resources (case loads for assistant 
prosecutors, for example), and statistical analysis (e.g., case disposition 
rates and case processing times). 

Automated PMIS procedures vary widely. The origination of data, for 
example, may be- accomplished in the same manner as in a manual system with 
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an extra copy of documents being prepared as the "source document" for 
entering data into the. computer system.. Data originated in this manner may 
be prepared for entry into the computer system by either "off-line" or 
"on-line" methods. Using the off-line method, source documents are normally 
delivered to a processing facility where data are prepared for entry into the 
computer by keying the data into punched cards or onto magnetic tapes, with- 
out involving the computer. On a scheduled basis, the batch of records thus 
generated are fed into the computer for processing, hence the term "batch 
processing".* On-line data entry may also be accomplished by sending source 
documents to a central data processing facility for keying directly into the 
computer. However, the most common method of on-line data input is to use 
remote data entry devices, usually located in the office originating the 
data. 

On-line operations are accomplished with terminals (keyboard devices), 
such as electric typewriters, that are connected to the computer either 
directly by a cable or from a remote location** by telephone lines. In 
either case, the terminals provide direct access to the computer system. 

The type of terminal employed will determine whether the computer is 
engaged for every terminal operation. Some devices, called "buffered" 
terminals, have the capability to store data within themselves and some, 
called "intelligent" terminals, have the capability to be programmed to 
perform basic functions such as data editing. "Intelligent" or "buffered" 
terminals do not engage the computer for every operation (e.g. , information 
may be stored in the terminal during data entry operations and subsequently 
fed, on-line, to the computer in a single data stream consisting of all the 
records stored in the terminal). Unbuffered terminals, on the other hand, do 
not have a storage capability and, therefore, engage the computer for each 
operation. 

The equipment configuration for terminals also varies widely. A common 
practice is to include with the keyboard a video display device (cathode ray 
tube - a TV screen) as a component of the terminal. This device is used to 
display formats (called "screens") that aid the terminal operator in entering 
data or aid in keying in commands to the computer to perform specific pro- 
cessing transactions (e.g., to inquire about the trial date for a defendant). 

* Batch processing, as used in this study, also refers to the processing 
that is accomplished at given intervals (e.g. , daily) whereby data that 
have been entered into the PMIS during a specific period of time (preceding 
24 hours, for example) have been sorted into groups (batches) before the 
processing of scheduled reports begins. Batch processing usually involves 
the manipulation of many records, in comparison with on-line processing 
which may involve retrieving only one or a few records; and, batch proces- 
sing is usually accomplished during a time period when on-line operations 
are minimal. 

♦♦Remote location, as used here, means that a terminal is too far from the 
computer to be connected by cable; it may be located in the same building, 
an adjacent building, or miles from the computer facility. 



Another terminal configuration may include an on-line printer. This config- 
uration permits the rapid generation of printed reports or documents in 
response to a request by the terminal operator. 




Use of an on-line printing capability allows variations to data entry 
procedures. As an example, rather than preparing source documents for use in 
data entry operations, as previously discussed, certain data or commands can 
be transmitted, on-line, to the computer from a remote terminal. The so- 
called source document can then be generated by the PMIS via the on-line 
printer (e.g., arrest reports using on-line booking operations, or generation 
of subpoenas, as performed by Norfolk's TRACER system). 

Various documents (such as arrest reports, prosecution work sheets, 
schedules, dockets, and disposition reports) are needed in hard copy form to 
satisfy prosecutor and court operating procedures and to comply with estab- 
lished recordkeeping requirements. In a non-automated system, all of these 
documents are prepared by manual methods, whereas in an automated system the 
extent of PMIS generation of such documents will depend on: what data ele- 

ments are entered into the system; and, what computer programs have been 
written to satisfy PMIS processing and output requirements. 

Computer programs are instructions, usually written in a standard pro- 
gramming language such as the Common Business Oriented Language (COBOL), that 
tell the computer how to handle and store input data, how to manipulate the 
data to satisfy processing requirements, and what outputs are to be generated. 
These programs, along with other documentation that describes system proce- 
dures, constitute the software for an automated PMIS. 

The data elements that may be entered into the PMIS consist of items 
such as: the defendant's name, age, sex, race; offense charged; date of 
offense; witnesses names, addresses, phone numbers; dates of hearings, trial 
and sentencing; and so forth. The number of different data elements, relevant 
to a specific defendant or case, entered into a PMIS depends largely on which 
criminal justice agencies are "users" of the PMIS. t 

A PMIS user, for the purpose of this study, is an agency that has direct 
access to the PMIS and is authorized to retrieve and/or update PMIS defendant 
or case records. Users, then, "share" the PMIS data. 

The extent of data sharing is a major PMIS characteristic and influences 
the number of different data elements stored in the system. A PMIS may 
support only the prosecutor. In this type of system only data of concern to 
the prosecutor are entered into the PMIS; ‘'items of interest to other agencies, 
such as the police (e.g., identification of stolen property) or the court 
(e.g., accounting for the collection of fines), would be excluded from the 
system. Another type of PMIS is one shared by the prosecutor and courts. 

The extent of data sharing is rather limited in this type of system, but the 
data elements would include items of interest to both the prosecutor and 
courts. In other words, the greater the number of system users, the greater 
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the number of data elements in the data base. An automated system, in fact, 
may include several or all criminal justice agencies within a jurisdiction. 

This type of system, called an "integrated system" in this study, would 
have specific files, within the data base, devoted to each user. Users, 
then, would update (add or change records) their own files; with appropriate 
interagency ageements,. they could retrieve information from other user files 
(or portions of the other files); but, in general, they would not be able to 
update the files devoted to other users. 

The capability to "talk" to computers via telecommunication facilities 
allows data sharing by more than one jurisdiction, and, if desired, can be 
accomplished on a statewide, national, or worldwide basis. 

The capability to talk to computers is not limited to man-machine communi- 
cations; computers can talk to one another (machine-machine communications). 
This type of arrangement, called a computer "network", is accomplished through 
telecommunications facilities and by the use of computer programs written 
specifically to control the network operations. 

Computer networks provide processing capabilities that are, obviously, 
beyond the realm of single computer operations. In the criminal justice 
community, for example, individual jurisdictions could have their own com- 
puters linked to a state level computer system. On a prearranged schedule 
(e.g., daily), the computer at each jurisdiction could automatically transmit 
selected data (e.g., warrants) about local offenders. If an offender from 
one jurisdiction is apprehended in another jurisdiction, the apprehending 
jurisdiction could make an inquiry to its local computer to determine the 
status of the offender. If the offender is not wanted in the apprehending 
jurisdiction, the local computer file will not contain the warrant informa- 
tion. The inquiry would then be automatically transmitted to the state 
system; the warrant information would be retrieved from the state system and 
automatically transmitted back to the apprehending jurisdiction; and the 
jurisdiction that issued the warrant could be automatically notified of the 
apprehension via the computer network. 

A computer network, such as described above, could also be used to 
process periodic reports at the state level, using the data transmitted 
regularly by individual jurisdictions. This arrangement would relieve the 
individual jurisdictions of those reporting requirements. An expansion of 
this approach could be the use of the state level system (or some centralized 
system) to perform the processing necessary to generate most of the hard copy 
documents required by individual jurisdictions, even those needed on a daily 
basis. This approach would reduce processing requirements within the indi- 
vidual jurisdictions, thus reducing the size requirement for computers at 
those locations. 



Many makes and models of computers' are available to support PMIS opera- 
tions. The predominant characteristic, however, is the computer size. A 
PMIS designed to support only the prosecutor, for example, may employ a 
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minicomputer or medium scale computer that could be housed and operated in 
the prosecutor's office complex. Another approach to a prosecutor dedicated 
system could be the use of a large scale computer located in and operated by a 
centralized city, county, or state data processing facility. In the latter 
situation, the computer system would have files devoted to the PMIS and 
accessible only by the prosecutor. 

A centralized, large scale computer would also be used to support an 
integrated system or a system designed to support multiple jurisdictions. In 
a computer network, as previously discussed, individual jurisdictions could 
be equipped with minicomputers that would be linked to a large scale computer 
at a central (state or regional) facility. 

Several characteristics can be used to describe the difference between, 
mini, medium and large scale computers. The most distinguishing character- 
istics concern physical size, costs, main memory capacities* and personnel 
resources required to operate the different systems. ^ (See Appendix A 
for references.**) 

Minicomputers are small in size and relatively inexpensive. Some are 
about the size of a desk and a few models (microcomputers) are small enough to 
place on top of a desk. In a typical minicomputer installation, two or three 
terminals may be used for data input and system control; one tape drive or one 
disk drive may be used for secondary data -storage; and a printer would be 
connected to the system to provide hard copy outputs. The main memory unit 
may contain from 4,000 to 512,000 storage locations. However, memory capac- 
ities are being continuously increased without increasing the physical size of 
the memory unit, through the advancement of miniaturization techniques. The 
cost of purchasing a minicomputer may range from $50,000 to $200,000 or one 
may be rented for $1,250 to $5,000 a month. Such a system could be operated 
by as few as one or two people. 



* Computer main memories (often called the "core") consist of electrical cir 
cuits within the central processing unit. The "on" or "off" condition of a 
circuit is used to represent the 0 (off) and 1 (on) digits in the binary 
numbering system. In computer jargon, a binary representation (the 1 or 0) 
is called a "bit". Combinations of bits (usually seven or eight) are used to 
code characters for storage in the computer memory (e.g., in one seven bit 
coding scheme, 1'OQOOOI = A). The circuits used to code a character (seven 
circuits in this example) are referred to as a storage "location". Memory 
capacities, as discussed here, are measured by the number of storage loca- 
tions contained in the core. 

♦♦Definitions of different size computers are taken from publications printed 
in 1977-1978. Costs included in these definitions may differ somewhat in 
today's market because of inflation, but the relativity of these costs 
between computers remain the same and they relate to the costs encountered 
during the surveys conducted for this study. 








t 



5 






i 



r 



r 



r 



r 



c 



r 









i? 



i 



Medium scale computers normally have the capability to handle more 
peripheral devices than minicomputers; their main memories have greater capac- 
ities; and their physical space requirements are somewhat geater than for the 
mini's. These computers may range in price from $200,000 to $1,000,000, or 
they may be rented for between $5,000 and $25,000 a month. Usually, four or 
five people, including a couple of programmers, are employed to staff a 
medium scale computer operation. 

Large scale computer facilities are typically laden with equipment and 
people. It is not unusual to find two large scale computers in one such 
facility with over 50 people employed to operate the equipment and to write 
computer programs. The extensive capabilities of these computers permit the 
use of many peripheral devices; the use of remote terminals, which may number 
over 100, requires special telecommunications equipment (modem and monitors). 
Thus, a large area is required to house a large scale computer and its periph- 
eral equipment, and separate office space is needed for the programming staff. 
Typical memory capacities range from 64,000 to 4,000,000 locations - some of 
the biggest systems have 30 billion locations. The costs of purchasing or 
renting large scale computers ranges widely, of course, depending on memory 
capacities and the equipment configuration. Some systems may be purchased for 
about $500,000 while others may range up to $3,000,000. Rentals may start 
around $12,500 a month and range to $50,000 or more. 

Hardware for an automated PMIS, then, consists of standard office equip- 
ment plus the equipment that makes up the computer configuration, typically: 

• A central processing unit (CPU) - the computer main frame - 
contains the main memory (storage) unit; 

• Secondary storage (tape and/or disk drives); 

• Input/output devices (terminals and printers, for example); 

■ Telecommunications monitors and modems (where remote terminals 

are used); and 

• Data preparation devices (keypunch or key to tape machines - 
where off-line data entry is employed). 

There is also a correlation between the size of a computer and the speed 
at which it operates. Large scale computers operate at speeds measured in nano- 
seconds (one-billionth of a second) while medium scale and minicomputer speeds 
are usually measured in microseconds (thousands of a second). 

Computer speed is an important factor when selecting a system to effi- 
ciently process very large volumes of data. Some organizations, however, may 
have a data volume that is insignificant as far as computer speed is concerned: 
it may not matter to a prosecutor, for example, whether a computer operates in 
nanoseconds or microseconds; what will matter is that a computer can effi- 
ciently process a volume of data that has surpassed the capability of the 
prosecution staff to handle efficiently using manual methods. 




Increased caseloads, resulting in increased data volumes, and the desire 
to improve the efficiency of case processing has prompted prosecutors across 
the nation to consider automation of their prosecution management information 
system. 

A variety of automated PMIS's have been developed and implemented over 
the past 10 years to assist prosecutors in their daily operations and in 
management of resources. Of these, the largest single program has been the 
Prosecutor's Management Information System (PR0MIS).* 

•PR0MIS was developed by the founders of the Institute for Law and Social 
Research (INSLAW) for the U.S. Attorney for the District of Columbia in 1971. 
The original PR0MIS was a batch processing system (no on-line operations) 
designed to run on a large scale computer. Subsequently, with LEAA funding, 
PR0MIS was redesigned as a transferable system and implemented in other juris- 
dictions, such as New Orleans and Indianapolis. In Milwaukee, Wisconsin, 
substantial modifications** were made to PR0M1S, and the modified system was 
then transferred to other jurisdictions, such as Louisville, Kentucky. 
Improvements resulting from redesign and modification included the addition of 
on-line capabilities and the development of additional software designed to 
operate on minicomputers. 

Two minicomputer versions of PR0MIS software have been developed: one, 

called MINI-PR0MIS, was designed to operate with unbuffered terminals; the 
other system, called MAXI-PROMIS or new-PROMIS, has been designed to use 
buffered terminals. The two minicomputer versions of PR0MIS can be run on 
large scale computers as well as on minicomputers, and both software packages 
feature a "tailoring" capability. 

The PR0MIS tailoring capability^ allows individual jurisdictions to 
design their own: data base (files and records); data entry screen and edit 

criteria; inquiry displays; indexes; and formats for output reports. This 
tailoring is accomplished by use of an on-line, interactive computer program 
(using a question and answer format) that takes the user, step-by-step, 
through a series of tables that define the files, transactions, data element 
characteristics, and so forth, as specified by the individual jurisdiction. 

MINI- and MAXI-PROMIS programs have been written in the COBOL programming 
language and have been designed to operate on a number of different computers. 



* For the purpose of this report, the acronym "PR0MIS" refers to the system 
developed for LEAA by INSLAW; the acronym "PMIS" refers to both PR0MIS and 
non-PRQMIS systems. 

**For the purpose of this report, modified versions of PR0MIS, such as Mil- 
waukee's JUSTIS, are called Hybrid versions of PR0MIS. This designation 
reflects inclusion of Milwaukee as an "operational" PR0MIS project in 
INSLAW publications, while LEAA officials and Milwaukee staff consider 
3USTIS as a non-PROMIS project. 
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These features, plus the tailoring capability, are intended to give the pros- 
ecutor maximum flexibility in designing system software and in the selection 
of hardware. 

In addition to the batch, on-line, hybrid, MINI-PROMIS and MAXI-PROMIS 
. versions, -manual and semiautomated versions of PROMIS also exist. As of 
-January 1980, INSLAW reported that: 37 jurisdictions are operating some 

version of PROMIS; 71 other jurisdictions are in the process of transferring 
PROMIS; and 63 other jurisdictions are planning for or evaluating PROMIS.^ 

One might expect that a system developed primarily by one organization, 
such as INSLAW's PROMIS, would be similarly configured in many of its opera- 
tional sites. In fact, PROMIS project configurations vary widely. In three 
PROMIS sites (Los Angeles, New Orleans and the District of Columbia, for 
example), data entry is accomplished in a batch processing mode with* on-line 
inquiry capabilities; two of those systems run on large scale, centrally 
located computers and share their data base with other users; the New Orleans 
system, however, is run on a large minicomputer that is operated and used only 
by the prosecutor. At other PROMIS sites (Louisville and Milwaukee, for 
example), large scale, county operated computers function in an on-line mode 
and the system is shared extensively by other users. Large scale PROMIS 
operations, such as Tallahassee, Florida, provide on-line support to multiple 
jurisdictions; and MAXI-PROMIS is being implemented in Colorado using a large 
scale computer to provide support on a regional basis. In Portland, Oregon, 
on the other hand, MAXI-PROMIS is being implemented on a large scale, county 
operated computer, but will be used only by the prosecutor. 

Non-PROMIS projects also vary widely, insofar as system configuration 
and extent of data sharing are concerned. In Oakland, California (the DALITE 
system) and Boston, Massachusetts (the Case Management System), for example, 
minicomputers operated by prosecution personnel are dedicated to functions 
of the prosecutor's office. A more common situation is a large scale computer, 
operated by a central city or county data processing facility, providing on- 
line support to the prosecutor and other system users. The extent of data 
sharing will vary among these centralized systems. In Baltimore, Maryland, 
for example, the system supports the courts and the prosecutor; other systems, 
such as those located in Miami, Florida, and in Fort Worth, Texas, support all 
criminal justice agencies throughout Dade and Tarrant countries, respectively; 
and, the PMIS in Norfolk, Virginia, supports criminal justice agencies within 
the Tidewater Region. 

The system used in Norfolk can also be used as an example of a computer 
network: the PMIS, called TRACER, interfaces with the Tidewater Electronic 

Network of Police Information (TENPIN) which, in turn, interfaces CPU to CPU 
with the Virginia Criminal Information Network (VCIN) and, in turn, the 
National Crime Information Center (NCIC). 

Another variation to large scale PMIS operations can be seen in Oklahoma 
City, Oklahoma where a county data processing facility provides support on a 
statewide basis: five counties operate terminals connected on-line to the 





computer for data input and for query/response transactions; the other counties 
send source documents to the Statistical Analysis Center (SAC) of the Oklahoma 
Crime Commission for input to the computer and are able to obtain responses to 
inquiries by telephone contact with SAC where the inquiry is made via on-line 
terminal and the response received by SAC is relayed verbally using the 
‘ telephone. 

Implementation of non-PROMIS systems has, for the most part, been based 
q.n independent development by individual jurisdictions; transfer of non-PROMIS 
systems is not a common practice.* Development of non-PROMIS systems and 
transfer of PROMIS have been accomplished by in-house personnel, contractors, 
or a combination of the two, depending on the availability of in-house person- 
nel with the appropriate expertise. 

C. Need for PMIS Evaluation 

Increasing criminal caseloads coupled with limited resources have moti- 
vated many prosecutors to consider automation as a means of improving manage- 
ment and operations. Indeed, over 175 of the larger prosecution offices 
surveyed by mail during this study (having more than 25 employees), either 
have an operating PMIS or are in the process of planning such a system. 

. Based on the survey data collected during this study, it is estimated 
that in excess of $30 million has been spent in the last decade developing 
various types of automated systems to serve prosecutors and over $20 million 
is spent each year on the operation of such systems. Until quite recently, 
the outlook was for substantial expenditures to continue due to increasing 
interest of -prosecutors in automation, the current funding policies of LEAA, 
and interest in criminal justice research. In light of diminishing criminal 
justice budgets, it is expected that the results of an evaluation of PMIS's 
will be useful to Federal officials and state planning agencies in assessing 
-PMIS funding policies, reviewing grant applications, and allocating funds 
among competing projects. Prosecutors, courts, other criminal justice agen- 
cies, and state and local government data processing service organizations 
should find evaluation measurements useful in identifying elements of PMIS's, 
and their associated implementation projects, that have been successful and 
•effective in improving prosecution and court performance. Prosecutors, 
judges, and prosecution/court administrators will gain insights into factors 



♦The LEAA 1976 Directory of Automated Criminal Justice Information Systems 
lists 24 systems with the function of "prosecution management"; of the 20 
non-PROMIS systems on this list, none are transferred systems. The Criminal 
Oriented Records Production Unified System (CORPUS), operated by Alameda 
County, California, was originally transferred from Santa Clara County, 
California, but extensive modifications have been made to the original 
system; CORPUS is the only transferred non-PROMIS system that was examined 
during this study. 
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contributing to PMIS success and failure, methods of measuring performance, 
and approaches to improving PMIS projects and prosecution/court performance. 
Information resulting from evaluations should also assist in determining 
changes or additions to make PMIS projects more effective. 



D. Purpose and Scope of the Study 

This NEP Phase I study was conducted in two segments. The first segment, 
performed during the period of October 1978 through November 1979, was 
designed .to identify and describe the nature and extent of prosecution MIS 
usage and the problems, costs and benefits of such use. This segment was also 
designed to determine if data are available to support in-depth, objective 
evaluations (NEP Phase II studies). Information needed to meet the study 
objectives of this first segment was gathered by means of a mail survey of 594 
prosecutorial agencies and by site visits to 17 locations where automated 
information systems used by prosecutors are currently in operation. Mail 
survey and site visit data were augmented by information obtained from avail- 
able literature (see Appendix B) and from telephone contacts with individuals 
throughout the judicial system. 

The second segment, initiated in January 1980 and concluded in June 1980, 
was designed to determine the extent to which prosecution management informa- 
tion systems can be objectively evaluated and to test evaluation methodologies. 
Site surveys of six PMIS projects were conducted for the purpose of satisfying 
the objectives of this study segment. These surveys included four PMIS 
projects that had been visited during the first- segment .of the study plus two 
other sites that were in the process of implementing the latest version of 
PRGMIS (MAXI-PROMIS) . 

Appendix C contains a list of personnel interviewed during site visits 
in both segments of the study. 



E. Documentation of the Study 

This study has been documented in a report entitled "National Evaluation 
Program Phase I Final Report, Prosecution Management Information Systems," 
which consists of three volumes: 

• Volume I - Evaluability of Systems - defines the objectives and 
scope of the study; outlines the evaluation framework used; describes 
various types of prosecution management information systems that were 
observed; discusses the st at e-qf-t he-art„and state of knowledge about 
these systems; and presents the findings from the first segment of 
the study regarding quantitative results, surveys of PROMIS projects, 
and the investigator's judgmental assessments of PMIS projects. 

• Volume II - Feasibility .Tests - covers the second segment of the 
study and defines the purpose of the tests and data collected; 






outlines the evaluation design; identifies the test sites and 
describes the criteria used for their selection; and discusses the 
• -"test results. 

a Volume III - Site Visit Reports - contains a case study of each 

project surveyed during the first and second segments of the Phase. I 
study. Each report describes the characteristics of the organiza- 
tion and PMIS surveyed, describes the judicial process within the 
jurisdiction, and discusses the findings, observations and investi- 
gator's judgmental assessments regarding the PMIS. 






10 



11 



II. EVALUABILITY OF SYSTEMS 



A. Introduction 

Since the characteristics and environments of PMIS projects vary widely, 
the first segment of this study was designed to survey these projects and 
develop comparable information within a structured evaluation framework. A 
major objective of this segment was to assess the evaluability of the various 
PMIS projects. 



B. Methodology 

Two primary techniques were employed in gathering information about what 
exists: (1) a mail survey with telephone followup, and (2) a set of site 

visits . 

1 . Mail Survey 

The sample for the mail survey was selected from a Bureau of Census 
tape file which contained a total of 3,715 agencies.* Excluded from the 
sampling process were agencies with less than five employees and agencies 
which: 

■ Provide legal services only; 

■ Deal exclusively with juvenile cases; 

• Deal exclusively with traffic violations; or 

/ 

• Are branch offices. 

After eliminating out-of-scope agencies that could be identified 
by data on the tape, the remaining 1,533 agencies were stratified by number 
of employees. All agencies in stratum A (25 or more employees) were 
included in the sample. Agencies in stratum B (10-24 employees) and C (5 - 
9 employees) were sequenced by level of jurisdiction (state, county and 
municipal) and by census division; choosing two sites at random out of each 
block of four sites on the list, half of stratum B was selected; and simi- 
larly every fourth agency in stratum C was selected. As shown in Table 1, 
the result was a sample of 699 agencies from which 105 were deleted as out- 
of-scope. The remaining 594 in-scope agencies formed the basis for the 
mail survey. 



♦State and Local Prosecution and Civil Attorney Systems, U, S. Department of 
Justice, March, 1978. 
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Stratum 


No. of 
employees 


No. of 
agencies 
selected 


Sample 

percent 


No. of 
agencies 
selected 


No . o " 
agencies 
out-of-scope 


No. of 
agencies 
in sample 


A 


25 or more • 


287 


100 % 


. 287 


47 


240 


B 


10 - 24 


400 


50 % 


200 


17 


183 


C 


5-9 


846 


25 % 


212 


41 


171 


Total 




1533 




699 


105 


594 



Table 1. Mail Survey Sample 



2. Site Surveys 

The following criteria were used to select projects for site surveys: 

a Automated system operational for at least three 
months; 

• Staff is cooperative; 

• Documentation and data are available; 

• Applications are of general interest; 

• Availability of data for performance measures; 

* 

• Project is innovative and widely applicable; 

• There is an interesting organizational aspect, such as 
regional time-sharing or an integrated police/prosecutor/ 
court/corrections interface; and/or 

• A significant number of prosecutor MIS applications are 
represented. 

Based on a review of available literature, 50 PMIS projects were 
selected as' prime candidates for on-site surveys. As a result of telephone 
contacts with site personnel and responses to mail surveys, a final selection 
of 17 locations for site visits was made. A list of these sites, and a 
description of the type of PMIS used is shown in Exhibit 1. 
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Exhibit 1. Projects Selected for Site Survey 



PROJECT NAME (ACRONYM) 

AND LOCATION (COUNTY) 

1. Prosecutor's Management 

Information System (PROMIS) 
Los Angeles (Los Angeles) CA 



Criminal Oriented Records 
Production Unified System (CORPUS) 
and District Attorney Automated 
Legal Information System (DALITE) 
Oakland (Alameda) CA 



3. Criminal Justice Information 
Center (CJIC) 

San Jose (Santa Clara) CA 



4. Superior Court/County Clerk 
Information System (SUPER/CC) 
Santa Ana (Orange) CA 



5. Automated Court Information 
System (ACIS) 

San Bernardino (San Bernardino) CA 



6. Dade County Criminal Justice 
• Information System (CJIS) 
Miami (Dade) FL 



7. Judicial Automated Records 
System (JARS) 

Waukegan (Lake) IL 



TYPE OF SYSTEM 

Integrated system 
Large scale computer 
Operated by county 
Limited data sharing 

CORPUS - Integrated system 

Large scale computer 
Operated by county 
Extensive data sharing 

DALITE - Prosecutor dedicated 
system 

Minicomputer 
Operated by prosecutor 
Access by prosecutor 
only 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system » 

Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
■Large scale computer 
Operated by county 
Extensive data sharing 
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Exhibit 1. Projects Selected 



PROJECT NAME (ACRONYM) 

AND LOCATION (COUNTY) 

B. Prosecutor's Management 

Information System (PROMIS) 
Indianapolis (Marion) IN 



9. Commonwealth Attorney's Tracking 
and Case History System (CATCH) - 
PROMIS 

Louisville (Jefferson) KY 

10. District Attorney's Record 

Tracking Systems (DARTS) - PROMIS 
New Orleans (Orleans Parish) LA 



11. Case Management System (CMS) 

Boston (Suffolk) MA 

| 

s. . 

I . 

1 ^ 12. Arrest Disposition Reporting 

System (ADRS) 

| Oklahoma City - Statewide OK 

j 

1 

13. Tarrant County Criminal Justice 

* Information System (TCCJIS) 

I Fort Worth (Tarrant) TX 



'14. Total Recall Adult Criminal 
; Element Record (TRACER) 

| ® Norfolk - Regional VA 

f 

) 

i 15. Justice Information System 

(JUSTIS) - PROMIS 

* Milwaukee (Milwaukee) WI 




for Site Survey (Continued) 



TYPE OF SYSTEM 

Prosecutor dedicated system 
Minicomputer 
Operated by county 
Access by prosecutor only 

Integrated system 
Large scale computer 
Operated by county 
Limited data sharing 

Prosecutor dedicated system 
Minicomputer (large) 

Operated by prosecutor 
Access by prosecutor only 

Prosecutor dedicated system 
Minicomputer 
Operated by prosecutor 
Access by prosecutor only 

Integrated system (statewide) 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system (regional) 
Large scale computer 
Operated by city 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 
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C. ' Evaluation Framework j, 

PMIS evaluation is based on the notion that a prosecutor has defined, at I 

least implicitly, goals for the operation of his or her office. These goals f, 

are normally defined in such terms as conviction rates, evenhandedness, delay, | 

or other attributes of the criminal justice system. ^ The PMIS is developed |j 

and operated in the hope that it will assist in the achievement of goals, i.e., lj 

that it will have a positive impact. A logically structured impact evaluation f 

must be carried out within a methodological framework that: I 

• Provides a general theory linking PMIS operation to impact on the | 

criminal justice system. 1 

• Defines prosecutor goals in terms of specific elements of PMIS 
impact . 

• Defines indicators of success — operational measures of these 

elements that can be used to quantify the extent to which PMIS 1 

operation has impacts that advance the prosecutor's goals. 

Figure 1 summarizes a theory of how PMIS development and operation are 
linked to impact on the criminal justice system. Ideally, development of a 
PMIS takes place in four steps: requirements analysis, system design, system 

•software development or transfer, and system implementation. Once the PMIS is 
operational, it may produce three categories of outputs: operational outputs , 

such as lists of witnesses, charging instruments, or subpoenas; records of 
operations and decisions such as in individual cases, attorney assignments, 
hearing dates or motions granted; and statistical displays in which these case- 
level operations are aggregated into such measures as dismissal rates, 
average delay, or case mix. 

In turn, these outputs may modify the prosecution or judicial process in f : 

a variety of ways. For example, witness lists, including accurate addressses *- 

and/or telephone numbers, may be used to administer the "on-call" feature of a 
witness assistance program to avoid unnecessary trips to the courthouse, only 
to learn that a hearing has been continued. Up-to-date and accurate records 
of attorney assignments and hearing dates may be used to reduce the incidence 
of schedule conflicts for individual assistant prosecutors. Statistical 
displays of case duration may be used to monitor 'Compliance with a speedy 
trial law. 

Such modifications of the process as these comprise the PMIS impact. In 
addition, by monitoring various indicators over time, the PMIS ideally pro- 
vides feedback not only on its own impact, but on the impact of external 

changes in the criminal justice process. ,! 

With the background provided by this overview of the theoretical linkage 
between PMIS operation and criminal justice system impact, it is possible to 
define more specifically the elements of PMIS impacts on prosecution and to 
relate them to goals of prosecution. Figure 2 identifies two general goals ^ 
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Figure 1 . Assunptions Linking Prosecution MIS to Impact on Prosecution/Court Criminal Justice System 
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PROSECUTION MANAGEMENT S 
ADMINISTRATIVE GOALS 


Utilize PM IS to 
more effectively 
manage & operate 
the prosecutor's 
office (9) 

■ 


Manage the PMIS 
project effec- 
tively (10) 
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PROSECUTOR OPERATIONS 
Goals 



M 

O 



Improve Schedul 
log a Logisti- 
cal Control 
(1) 



Reduce avoid 
able prose- 
cutor-re- 
lated post- 
ponements 

Improve com- 
munication 
with wit- 
nesses and 
court 

Monitor 
timeliness 
of disposi- 
tions 



Maintain Ef- 
fective Con- 
viction Rate 
( 2 ) 



Ma intain 
(Evenhandedness 
( 3 ) 



• Reduce in-| 
efficiency] 
related 
dismissals! 
& nol 
pros 

• Control 
plea nego-] 
tiations 

• Vigorously) 
prosecute 
cases with| 
appropriate 
priorities! 



• Maintain 
uniform 
chdrging 
and dis- 
cretionary! 
practices 

• Monitor 
prosecu- 
tor's de- 
cision 
patterns 



T 



Use Resources 
Efficiently 
( 4 ) 



Monitor at 
torney as- 
signments 
and uti- 
lization 

• Streamline 
administra' 
tive prac- 
tices 



FRAMEWORK 

ELEMENTS 



PROSECUT 


ukb MANAGEMENT INFORMATION SYSTEM (PMIS) 
Goals ~~ 

— 


Improve Information 
System Outputs 
< 5 > 


Improve Data 
Handling 
(6) 


PMIS Cost Control 
(7) 


Enable Resoarch/EvnlunL Ion 
Capability 
(8) 


• Query responses 

• Reports 

• Working 
documents 


• Collection and 
verification 

• Processing 

• Maintenance and 
update 

— H 


• Design 

• Acquisition 

• Operation 


• Offender tracking across 
cases am! agencies 

• Maintenance of historical 
data files with longitud- 
inal tracking capabil- 
ities 

• Transaction based case 
tracking with many 
descriptive variables 
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PHIS Project Actions to: 
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* Support Prosecutor Actions 
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Figure 2. Evaluation Framework 
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assumed to be important to the prosecutor implementing a PMIS: to manage 

the PMIS project itself effectively, and to use the PMIS to improve office 
management and operation. The former general objective is broken down 
into four more concrete prosecution goals: improving information system 

outputs, improving data handling, controlling PMIS cost, and developing a 
research and evaluation capability. The latter objective is broken down into 
four operational goals: improving scheduling and control, maintaining an 

effective conviction rate, maintaining evenhandedness , and using resources 
efficiently. 

Assuming that the prosecutor takes actions to manage the PMIS project 
effectively and to use PMIS outputs to satisfy his management goals, the third 
function of an evaluation framework is to define indicators of PMIS success. 

In other words, the evaluator must define operational measures that can be 
used to quantify the extent to which the PMIS advances the prosecution goals 
identified in Figure 2. These indicators are defined in Section II, Volume I 
of the Final Report. 



D. Analysis of PMIS Usage 

Three size groupings of prosecution offices were surveyed by mail in 
•August and September 1979. A comprehensive telephone survey was conducted in 
October 1979 to tabulate a minimal set of characteristics of the non-responsive 
prosecution offices. From this telephone survey, it was determined that 
certain offices in the sample were actually out-of-scope, according to the 
survey criteria. It is apparent that the smaller the office, the less likely 
it was to respond, possibly due to a lack of staff time or a lack of interest 
in the subject matter. 

The response rates are shown in Table 2. To what extent are any statis- 
tics developed from this set of responses indicative of the universe of prose- 
cution offices in the nation? First of all, the data should be analyzed by 
size group due to the differences in rate of response as well as expected 
need for an MIS. Second, researchers should be aware of any systematic bias 
that might exist in the responding population. Based on a comparison of the 
characteristics of the non-responsive group to those of the responding group, 
there does not appear to be an appreciable non-response bias in the propor- 
tions who have or plan to have an MIS, nor between PROMIS and non-PROMIS 
sites. 

The statistical tables that follow, then, should be understood for what 
they are. They present evidence that may be considered to support or refute 
certain hypotheses; they indicate patterns of PMIS use, problems, costs, and 
benefits, and comparable characteristics. They do not purport to constitute 
a representative sample of the population of prosecutors' offices, sufficient 
to make statistical inferences concerning the universe at large. Given the 
limitations inherent in that disclaimer, the data have been analyzed in 
various formats to yield insights on issues relevant to this study. 
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Information gathered during the 17 PMIS site surveys substantiate the 
fact that PMIS environments and characteristics differ in each jurisdiction. 

A detailed description of each surveyed PMIS project is contained in Volume 
III of the Final Report. 

The analysis and evaluation of individual PMIS support functions is an 
NEP Phase II task. PMIS support, for the purpose of this Phase I effort, has 
been synthesized to illustrate PMIS relationship to the judicial process, to 
compare prosecutor controlled systems with integrated systems, and to describe 
the types of output reports that support operational and managerial functions. 

The first analysis to be presented will indicate the nature and scope of 
PMIS use throughout the nation, based on data from the mail and telephone sur- 
veys^ The second analysis addresses the nature of PMIS support , based on the 
combination of PMIS functions observed during the site surveys. 



Table 2. Adjusted Response Rates 





Number 


Question- 




Number 


Adjusted 




of 


naires 


Total In 


of Valid 


Percent 


Strata 


Employees 


Sent Out 


Scope 


Responses 


Response* 


A 


25 or more 


287 


240 


109 


45 


B 


10-24 


200 


183 


55 


30 


C 


5-9 


212 


171 


45 


26 






699 


594 


209 


35 



♦Based on in-scope agencies. 

■Source?, PMIS Assessment Mail/Telephone Survey, Westat, Inc., August/September 
1979 



1 . Nature and Scope of PMIS Use 



The first set of findings from the mail survey describe the extent 
to which prosecution MIS's are being used or planned. 

As Table 3 shows, MIS's are used predominately by the larger offices. 
It is interesting to note that the proportion of MIS's planned in the smaller 
offices is substantially greater than the proportion used, which may indicate 
the effect of the trend toward smaller and less expensive computers. 
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The extent to which automation has been adopted by the nation's 
larger prosecution offices is impressive. Of 109 prosecution offices with 25 
employees or more, approximately 80 percent either use or plan to use an MIS. 
On the other hand, only about 19 percent of the smaller offices either use or 
plan to use an MIS. 

Comparisons of jurisdictions using an MIS with those planning an MIS 
reveal a substantially higher proportion plan to use PROMIS than currently 
use PROMIS. Of 43 jurisdictions that reported having a prosecution MIS, 37 
percent had some version of PROMIS; of 63 reported planning a prosecutor MIS, 
70 percent were planning to install some version of PROMIS. This may reflect 
the trend in LEAA funding policy or the effect of a recently introduced 
minicomputer version of PROMIS. These possibilities are examined later in 
this report. 

In order to obtain insights to the nature of the various prosecution 
MIS projects, a question was asked concerning the types of systems that 
prosecutors use or plan to use. Of 43 responding prosecution offices having 
an MIS, about 23 percent report using a minicomputer; while of 60 offices 
planning an MIS, about 40 percent report planning for a minicomputer. This 
indicates a trend toward the use of minicomputers. 



Table 3. Proportion of Prosecutors' Offices Using 
or Planning an MIS (PROMIS/Non-PROMIS) 
(percentages in parentheses) 




Source: PMIS Assessment Mail Survey, Westat, Inc. July, 1979 
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Since the number of prosecution offices, with less than 25 employees, 
that have or plan to have an MIS is too small to analyze further, the remain- 
der of this analysis will, focus on the larger prosecution offices, i.e., those 
with 25 employees or more. 

A question was asked concerning the jurisdictional area served by 
the prosecutor. The results show that about three-quarters of the responding 
offices are county prosecution offices. 

What prosecution goals are relevant to the MIS project, and what 
applications are planned or in ude in prosecutor MIS projects? The ratings 
reported by 100 of the larger prosecution Offices responding to the mail sur- 
vey indicated that the highest priority goals relevant to a prosecution MIS 
are control of scheduling and logistics, and allocation of staff based on 
prosecution priorities. Any assessment of PMIS projects should consider the 
capacity and impact of the MIS relative to these goals. Case status reporting 
and workload reports were the two applications most often in use or planned 
by prosecutors, followed in popularity by calendaring and scheduling capa- 
bilities. 

Due to the interest expressed by LEAA, a special telephone survey 
was conducted of PROMIS projects to supplement the data obtained in the mail 
survey. Information was obtained on sites contained on an INSLAW provided 
list as well as sites included in the mail survey. A total of 100 sites, 
thought to be operating, installing or planning PROMIS, were surveyed. Of 
these, 78 sites reported involvement with PROMIS. 

The PROMIS transfer program has been successful in promoting and 
accelerating the adoption of automated MIS capabilities in prosecutors' 
offices. The May 1979 user's conference was attended by over 260 people, 
representing over 75 user jurisdictions in varying stages of PROMIS transfer. 
.Since LEAA funding support for prosecution/court information systems now 
strongly favors the use of PROMIS, this program can be expected to grow in the 
future. „ 

Several versions of PROMIS have evolved, from the original batch 
version to the latest buffered or MAXI version. Substantial modifications 
have been made in certain implementations. It is, therefore, difficult to 
-discuss PROMIS as a single system; one must consider the various versions of 
PROMIS in any evaluation. 

More PROMIS systems (taking the various versions of PROMIS as a 
group) are planned or have already been implemented than any other single 
approach. These projects have involved transfer at the software level to a 
variety of hardware and jurisdictional environments. Evidence of the 
increasing trend in the adoption of PROMIS projects was revealed in the mail 
survey. 

Table 4 shows the overall status of the surveyed sites as of October 
1979 broken down by the various PROMIS versions. For the purpose of the survey, 
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PROMIS versions refer to the source of the original software package, and are 
defined as: 

• PROMIS I - the basic batch system provided by INSLAW; 

• PROMIS II - the basic batch system with an on-line inquiry capa- 
bility provided by INSLAW; 

• MINI-PROMIS - unbuffered terminal version, provided by INSLAW; 

• MAXI-PROMIS - buffered terminal version, provided by INSLAW; 

• HYBRID - a combination of PROMIS versions or a PROMIS version 
that has been extensively modified (e.g., JUSTIS - the large 
scale, on-line system developed by Milwaukee, transferred to 
and modified at other locations). 

Among operational PROMIS projects, the version most often used for 
transfer was PROMIS II, which combines batch updating with on-line inquiries. 
The second most popular operational version is Milwaukee County's 3USTIS, a 
highly modified version of the original batch PROMIS. JUSTIS was modified to 
operate on-line in a shared prosecution/court environment. The version of 
PROMIS most in demand by those prosecutors planning for an MIS is the MAXI- 
PROMIS. 









. °? ly 0ne °P erational system (Sacramento, California) was reported 

for the MINI-PROMIS version (unbuffered terminals) released in December 1978. 
Because of the desire for buffered terminals and the tailoring features 
planners of PROMIS projects have opted for MAXI-PROMIS. " " r 




All PMIS components (hardware, software, procedures, and types of 
people who operate the system) differ among all jurisdictions surveyed for 
this study. PMIS support functions also differ, but the primary difference is 
the number of functions supported by the PMIS; the purpose of a PMIS support 
function is usually the same among the various jurisdictions (e.g., in all 
jurisdictions where the PMIS produces prosecutor's caseload reports, those ’ 
reports are used by managers as an aid in assigning new cases) . 

, .. . The overall nature of PMIS support can best be described by combining 

I synthesizing) the various support functions that have been observed, reqard- 
less of individual PMIS environments and characteristics. 

Figure 3 is a synthesized version of felony and misdemeanor caseflow 
observed during the site visits. It shows the points within that process 
where MIS functions occur and where performance measures may be made. 

Tn the 17 jurisdictions visited, 14 of the information systems were 
shared by more than one criminal justice agency, and 11 were shared by all 
vor almost all) of the criminal justice agencies within the jurisdiction. 

F tt C »^n ldere ? a PP r °P riate to describe and analyze the prosecutor's segment 
of the MIS as it relates to the overall criminal justice system rather than 
confining the analysis to the prosecution functions per se. 

Exhibit 2 shows the relationship of MIS capabilities to the staqes 
of prosecution: y 

• t Events in the process — actions taken and decisions made; ^ 

» MIS activities — functions performed using an automated system; 

• Data recorded — the type of information entered into the MIS; 

• Performance measures — measures of effectiveness that can be 
taken provided the appropriate data have been entered into the 
system; and 

• MIS impact results that can be achieved by effective analysis 
of the performance measures. 

As mentioned earlier, the primary differences in PMIS support func- 
tions are the number of functions supported by the PMIS. These differences 

by f? mpa ^ n9 3 s y nthesized > integrated criminal justice information 
system (Figure 4) with a prosecutor controlled information system (Figure 5). 



* ill ■■ 

■■'A [ 



26 



T 




n 



\ 



c 



• ft 




ft f. 




I ft 


t s 




§ ft 


Exhibit 2. MIS Relationship to Judicial Process 


• 






A 




B 




C 


D 




I. 


EVENTS IN THE PROCESS 




MIS ACTIVITIES 




DATA RECORDED 


PERFORMANCE MEASURED 




Mir. liiPrtt’l 


1. ARREST AND BUDKINS 
















• Suspect arrested 


ft 


Inquiries 


ft 


Person/stntus 


• Arrest rates, by 


ft 


quicker persons 


• Suspect identified 




- Criminal history 




- Defendant, name, oe*, 


- Type of charge 




ident 1 1 lent inn 


a Arrest report prepared 




(local, State, 




DUB, address, employ- 


- Police officer 




- Posit tvo III 


• Suspect booked or 




notional) 




ment, location (jail, 


- Geographical area 




- Career Criminals 


released 




- Fingerprints (IocbI) 




bail) 


• Ball decisions, by 




- Across Jurisdirt lens 


• Bond henrinq - bail 




- Wants, warrants 




- Co-defendants (same 


- Type of charge 


ft 


Improve police prosecutor 


granted or denied 




(local, State, 




ns above) 


- Magistrate 




coon! lost ioo/cooperat ioi 


• Suspect jailed or 




national) 




- Witnesses 




ft 


Improve monitor evenhand- 


released 




- Probation 




- Bondsman 






(since!) ( ha 1 1 bond pn 1 i- 




ft 


DbIb entered 




- Arresting Officer 






cien and uni form 




ft 


Outputs produced 


ft 


Offense 






charging) 






- Arrest report 








ft 


Ident i 1 y liii|h crime. 






(booking packet) 










specific crime areas 






- Arrest warrant 








ft 


Reduce clerical 1 ssl n 






(commitment order) 








ft 


Minimize duplicate data 
















recording 


2. SCREENING 














■ 


• Review police report 


ft 


Inquiries 


ft 


Added, changed or 


• Nol pros rotes, by 


ft 


Improve - moo it or even- 


• Interview arresting 




- Criminal history 




deleted charges 


- Reason 




iiandednoee.i mini mm 


officer 




(local, State, 


ft 


Nol pros and reason 


- Charge 




charging pul iries • 


> ■ Add, change or drop 




national) on 


ft 


Prosecutor's name 


- Prosecutor 


ft 


Monitor screening puli- 


charges 




defendant, co- 




assigned to 


• Case filing rates, by 




cies in relation to 


• Evaluate witnesses 




defendants, and 




COSO 


- Reason 




case hacking'. 


i Accept or decline 




witnesses 


ft 


Victim/witness data 


- Charge 


ft 


Reduce clerical taoT'.i 


prosecut ion 




- Defendant status 






- Prosecutor 


ft 


Minimize ilopl leal e dal a 


• Identify cases for 




(jail, bail, etc.) 






■ Backlog measures 




recording 


special handling 


ft 


Data entered 














ft 


Outputs printed on-line 
















- rap sheet 














ft 


Outputs printed off-line 
















- Disposition reports 

















6z; 



T 



Exhibit 2. MIS Relationship to Judicial Process (Continued) 



3. FIRST COURT APPEARANCE 

• Boil decisions (pre- 

trial release or 
detention 

• Diversion programs 

• Charges dismissed 

• Date set for next 

court appearance 

• Appointment of 

defense attorney 



A. . PRELIMINARY HEARING (FELONIES) 

• Case presented • 

(partial ) 

• Charlies dismissed 

• Case dismissed 

• Charges reduced, re- 

ferred to mis- 
demeanor court 
a Case referred to Grand 
Jury 

a Grand Jury waived, case a 
scheduled for arraign- a 
merit 



Inquiries 

- Defendant tracking 

(status, bond 
information) 

- Criminal history 

- Schedules for 

judges, court- 
rooms, police, 
attorneys 
Data entered 
Outputs produced 

- Disposition reports 



Inquiries 

- Cose status (Case 

age) 

- Defendant tracking 

- Criminal history 

- Schedules for Grand 

Jury, judges, court- 
rooms, police, 
attorneys 
Data entered 
Outputs produced 

- Disposition reports 



Diversion program data 
Bail decisions 
Charges dismissed and 
reasons 

Dute of next court 
appearance 
Defense attorney's 
name 



• Charges dismissed/ 

reduced and reasons 

• Date of next event in 

the process 



■ Bail decisions, hy 

- Charge 

- Judge 

• Dismissal rates, by 

- Charge 

- Judge 

• Attorney, judge, court- 

room assignment 
statistics 



DismiBsal/reduct ion 
rates 

- Charge 

- Judge 

Attorney, judge, court- 
room, Grand Jury 
assignment statistics 

Case aging 



Improvo/muuiUir ovenhand- 
odncmi (uniformity in 
bail ducisiuns and 
dismissals) 

Minimize schedul ing 
cunf l icl a 
Reduce recidivism 
Reduce case processing 
time 

Reduce clerical tasks 
Minimise duplicate 
datu recording 



Improvu/monitor even- 
handedness (uniform- 
ity in dismissals 
und reductions) 
Minimize scheduling, 
conflicts 

Reduce case processing 
t ime 

Reduce clerical tasks 
Minimize duplicate data 
recording 



GRAND JURY/F1 LINGS 

• Case presented (suf- 

ficient evidence 
to indict) 

a Charges accepted or 
changed 

• Indictment (true bill) 

or refuse to indict 
■ Informations filed 



Inquiries 

- Charging prose 

- Case status 

- Defendant tracking 

- Cases scheduled for 

Grand Jury 
Data entered 
Outputs produced 

- Cases scheduled for 

Grand Jury 

- Grand Jury selection 

- Disposition reports 



Charges changed and 
reasons 

No indictments and 
reasons 

Date of next court 
event 

Judge/court-room 

assignment 



Indictment/no indictment 
rates, by 

- Charge 

- Attorney 
Case aging 



Evaluat e/ improve at t ur- 
ney performance 
Reduce case processing 
t ime 

Reduce clerical tasks 
Minimize duplicate data 
recurding 
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Exhibit 2 MIS Relationship to Judicial Process (Continued, 
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events in the process 



6. ARRAIGNMENT 



Defendant arraigned 
Mot Iona heard, 
accepted or 
rejected 

Charges accepted, 
reduced or 
dismissed 
Trial date aet 



7. TRIAL 



Ed 

O 



Q\\ 
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Defendant tried by 
judge or Jury 
Guilty plea accepted 
or rejected 
Attorneys present 
evidence 

Witnesses testify 
Verdict reached 
Dnte set for 
sentencing 



8. SENTENCING 



• Arguments presented 

• Sentence invoked 



HIS ACTIVITIES 



Inquiries 
“ Cose status 

- Defendant tracking 

- Cases scheduled for 

arraignment 

- Attorney's schedules 
Data entered 
Outputs produced 

- Cases scheduled for 

arraignment 

- Schedules/dockets 

- Witness subpoenas/ 

notices (for trial) 

- Disposition reports 



Inquiries 

- Case status 

- Defendant tracking 

- Criminal histories 

- Schedules 
Data entered 
Outputs produced 

- Schedules/ docket b 

- Disposition reports 



9. POST-SENTENCING 

» Incarceration, fine 
and/or probation 



Inquiries 

- Criminal history 

- Defendant tracking 

(for the already 
served, if not 
on release) 

- Case status 
Data entered 
Outputs produced 

- Commitment order 

- Disposition reports 



Inquiries 

- Defendant tracking 
Data entered 
Outputs produced 

- Jail/prison population 

reports 

- Time payment bills 

and 



OATA RECORDED 



Motions Bnd results 
New arraignment date 
(continuance) 
Charges reduced/ 
dismissed and 
reasons 
Trial date 



Guilty pleas 
Verdicts 
Date set for 
sentencing 
Mistrials and 
reasons 



Sentencing decision 



Name or corrections 
faci 1 Ity and cell 
assignment 

Amount 0 r Tine, t imn 
payment arrangement s 
Name uf pruimt inn 
officer 



PERfOltMANCt: MEASURES 



MIS IMPACT 



Postponement rates, by 
r Charge 

- Judge 

- Attorney 
Dismissal rotes, by 

- Charge 

- Judge 

- Attorney 
Cose aging 
Plea rates 
failures to appear 



Conviction rates, by 

- Charge 

- Judge 

- Jury 

- Attorney 
Case aging 
Disposlt ion rates 
Cnreer criminal 

slot istics 

Recidivism statistics 
Plea rates 



• Sentencing decisions, by 

- Charge 

- Judge 

s Case aging 



Probation violations, by 

- Charge 

- Defendant nodal 

characteriat tea 

- I'rolmt ion off icar 
failures to pay fines, bv 

- Charge 

- Defendant social 

charnel erisl ics 



Improve/mon i t or cvenhaud- 
ndnona (uniformity i„ 
dismissals and ledur- 
tinns - plea bargaining 
Evaluate improve judge 
at tornev perfoimauce 
Reduce cane processing 
limn 

Reduce clerical tasks 
Minimire duplicate data 
recording 



Evalunte/imprnve Judge, 
attorney performance 
Reduce pretrial case 
processing t imr 
Reduce clerical tnskH 
Minimize duplicate 
data recording 



Improve/mon i l or evenhnnd- 
edness (uniformity in 
sentencing) 

Reduce case processing 
t ime 

Reduce clerical tasks 

Minimize duplicate dsta 
recording 



Improve/monitor proha- 
I Inn policies 
Improve /man i t or po I lr ion 
for time payments ol 
fines 

Improve man I tor con- 
dil ions in correct ions 
fad 1 il ips (by prisoner 
I rans I era, 



NATIONAL CRIMINAL 
INFORMATION CENTER 



On-Line Booking 
Fingerprint Matching 
Suspect Tracking 
Criminal History 
Case Status 



Case Status 
Defendant Tracking 
Case Scheduling 
Management Statistics 
Criminal History 



STATE CRIMINAL INFORMATION 
CENTER 



LOCAL CRIMINAL JUSTICE 
INFORMATION SYSTEM 



On-Line Transactions 

- Response to Inquiries 

- Master File Updates 

- Index Updates 

- Updates of Other On- 
Line Files 

Off-Line Transactions 

- Outputs Supporting 
Operations 

- Outputs Supporting 
Management 



NUMBER 




f Z. 

_ COURT ^ 


, INDEX _ 




PROBATION 


_ POLICE 


• 


OTHER 


L Jail J 




AS NEEDED 



COURTS 


• 


Case Status 


■ 


Defendant Tracking 


• 


Schedules 


• 


Dockets 


■ 


Minutes 


9 


Management Statistics 
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Criminal History 



CORRECTIONS 



Inmate Tracking 
Management Statistics 
Criminal History 



PUBLIC DEFENDER 


/ v — 1 


K \ 


PRQBAT ION 1ST 


• Case Status 

■ Defendant Tracking 

• Case Scheduling 
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• Prabationist Tracking 

• Criminal History 

t 



MANAGEMENT All 

CJ 

OPERATIONAL AGENCIES 



On-Line Files 



HISTORY 1 
TAPES 1 



Ofr-Line 

Files 



BATCH 

REPORTS 
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Synthesized Integrated Criminal Justice Information 
System 









NATIONAL CRIMINAL 
INFORMATION CENTER 



• Criminal History 



STATE CRIMINAL INFORMATION 
CENTER 



• Criminal History 



ARREST 

WARRANTS L*J>olice 



• Case Status 
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• Criminal History 



• Case Status 

• Defendant Tracking 

• Case Scheduling 

• Management Statistics 



PUBIC DEFENDER 



• Case Status 

• Defendant Tracking 

• Case Scheduling 



PROSECUTOR MANAGEMENT 
INFORMATION SYSTEM 



On-Line Transactions 

- Response to Inquiries 

- Master File Updates 

- Index Updates 

Off-Line Transactions 

- Outputs Supporting 
Operations 

- Outputs Supporting 
Management 

- History Updates 
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Fiqure 4 represents an integrated system where all criminal justice 
agencies in the jurisdiction are users of the system; and, the system is 
part of a network that links the local computer with computers at the state 

and national levels. 

Fiqure 5 represents a prosecutor controlled system where only the 
prosecutor's office has direct access to the PMIS; the prosecutor and other 
agencies may have access to state and national level systems, but the local 
PMIS is not linked to those systems. 

Some prosecutors believe they are best served by a system under 
their own control. They believe that their data collection procedures would 
be more reliable and they could maintain and improve the programs as needed. 

The advantages and disadvantages to this form of organization are discussed 

next in Section II E. 

Differences between the integrated and prosecutor controlled systems 
concern, primarily, information exchange , data recordi ng., and MIS processi ng 

Informat ion Exchange . Integrated systems allow the updating of cen- 
tralized records as events occur throughout the judicial system* data- are 
immediately available to all system users and could have an impact on decisions 
made at each stage of the process. In the prosecutor controlled system, in- 
formation about the status of individuals, such as outstanding arrest warrants, 
escapees, and persons on probation, is disseminated by the res pons: ible agencies 
via manually prepared printed reports. Case tracking, defendant tracking, 
person tracking is accomplished through the automated system by the prosecutor; 
other agencies requiring updated information must communicate via telephone 
and resort to manual records for data retrieval. 

Data Recording. In integrated systems, user agencies enter^their 
subject-specific data into the automated system. A prosecutor controlled 
system generally has line prosecutors recording data, or clerks recording da 
from prosecutors' notes. Clerical staff generally enter data on events outside 
their purview (e.g., arrest report data). 

MIS Processing. The integrated system provides both on-line and 
off-line support to daily operations and management functions of all user 
agencies. The system controlled by the prosecutor provides on-line processing 
support to the prosecutor; off-line processing support is 'limited to outputs 
that can benefit the operations and management functions of only the prose- 
cutor and the court (schedules and statistics, for example). 

The integrated system generates reports to support the operations and 
management of all user agencies. Reports generated by a prosecutor controlled 
system are applicable to the operations and management of the prosecutor s 
office and the courts. The capability exists in both systems, however, to 
generate the same types of reports for the prosecutor and the courts. A lis 
of these reports is shown in Exhibit 3. 
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A. Calendars/Dockets/Schedules 

1. Cases Scheduled for Preliminary Hearing, by: 

• .Court/Judge 

• Prosecutor 

2. Cases Scheduled for Grand Jury 

3. Cases Scheduled for Trial, by: 

o Court/Judge 

• Prosecutor 

■ Defense Attorney 

4. Subpoenas/Appearance Notices 

• Witness Supoenas 

• Notices to Arresting Police Officers 

• Notices to Defense Attorneys 

• Notices to Bondsmen 8 

B. Case Status 

1. Master Lists of Pending Cases, by: 

• Case Number 

■ Defendant Name/Status of Defendant 

• Type of Case (felony or misdemeanor) 

2. Case Aging Lists - showing oldest cases first, by: 

• Court/Judge 

• Prosecutor 

• Pending Grand Jury Action 

3. Disposition Reports, by: 

• ' Court/Judge 

• PTosecutor 

• Defendant 

• Case Number 

C. Case Load 

1. Number of Cases Assigned, by: 

• Court/Judge 

■ Prosecutor (individually or grouped by prosecutorial unit) 

Exhibit 3. MIS Generated Reports 
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2. Specially Assigned Cases, by: 

• Court/Judge 

• Prosecutor 



D. Research and Analysis 



1. Caseflow analysis 



• Number of Bookings 

• Numbers of Cases Accepted for Prosecution 

• Dismissal Rates/Reasons for Dismissal 

• Conviction Rates by Type of Offense 

• Sentences by Gravity of Crime 

• Probation Rates 

• Incarceration Rates 



2. Work Load analysis 



• Caseload by Court 

• Caseload by Prosecutor • 

• Caseload by Type of Offense (felony and misdemeanor) 



3. Defendant analysis, by: 



• Age 

• Sex 

• Race 

y Offense 



€ 



€ 



e 



Exhibit 3. MIS Generated Reports (Continued) 
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E. Issues and Assessments 

Research conducted in this study has indicated certain primary areas of 
concern in t.he development, implementation, and operation of prosecution 

management information systems. These primary areas are: 

* * / 

• Iss ues Concerning the Organizati onal Context of a PMIS Project . 

These issues are concerned' with (a) whether the PMIS should be controlled by 
X" priUcSor or be part of a state, city or county court or orrmnal justice 
information system shared with other criminal justice agencies, (b) resource 
requirements for system development and improvement; and (c) the mtra- and 
inter-organizational relationships required to effectively manage a PMIS 
project. 

• Issues Conce rning the Development and Operation of a PMIS. These 
issues concern (a) the identification of management information requirements; 

(b) the evaluation and selection of alternative hardware and software 
approaches; (c) the staffing required for effective systems development and 
operation; and (d) procedural controls required for effective system opera- 
tion. • 

• Issues Concerning the Impact of a PMIS on the Cr iminal ^^£ 5. 

System. These issues are concerned with (a) the extent to which PMIS s meet 
their goals and provide their intended capabilities; (b) the impact of the 
PMIS on prosecution/ court processes and personnel; (c) the impact of a systeiv 
on prosecution performance; and (d) the evaluation of such impact. 

. Issues Co ncerning the Evaluation of PROMTS Projects . These issues 
are concerned with (a) the comparability and evaluability of PROMIS projects, 
(b) cost effectiveness of PROMIS projects; and (c) expectations of PROMTS 

projects. 

Each issue will be analyzed from three perspectives: first, the issue 

will be defined and discussed; second, where survey data applicable to that 
issue are available, they are summarized; third, the results of all aspects 
of data analysis are interpreted in making a judgmental assessment. 

1. Issues concerning the organizational context of a PMIS Project. 

ISSUE: Are there significant advantages to the p rosecutor control- 

ling his own system, as compared to partici p ating in an interagency s ha re, 
criminal justice information system? 



A crucial issue that has emerged in this study, one that has gener- 
ated wide differences of opinion and wide variations in project organization, 
is to what extent should the prosecutor control the collection, processing 
and reporting of prosecution data? There are at least four types of project 
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organization that have been identified in the literature and observed in the' 
f ield: 

• Computer operations and data base controlled by local 
prosecutor; 

• Computer operations and data base controlled by a statewide 
prosecution/court agency; 

• Computer operations centrally controlled by local government 
data processing agency; data base extensively shared by local 
criminal justice agencies; and 

■ Computer operations centrally controlled; data base controlled 
by local prosecutor or court; limited data sharing. 

Among the advantages cited for a self-contained computer facility 
controlled by the prosecutor are assured access and operation, priority on 
programmer expertise, and total security of the case files. Security and 
privacy could be controlled under a system shared by only criminal justice 
agencies; however, most prosecutors' offices are headed by elected officials, 
and performance measures can be politically sensitive. 

Polansky points out that county commissioners and state legislators 
are usually not happy about a court (or any other agency in their jurisdic- 
tion) operating an independent facility. ^ Independent computer operations ■, 
can lead to costly redundancy in hardware and staffing. 

Closely related to the issue of degree of system sharing is the 
issue of resources required for developing and operating the PMIS. Resources 
are limited and one of the major reasons for sharing the system with other 
county or state agencies is to keep the costs down. Most county data pro- 
cessing operations cannot afford a separate criminal justice unit for pro- 
.gramming or operation. Due to these budgetary constraints the prosecutor will 
have to wait until other possibly higher priority systems (e.g., tax collec- 
tions) are completed. This can be particularly frustrating when waiting for 
improvements in an existing PMIS. 

Management of the PMIS project is a complex task requiring coordina- 
tion' among all units of the prosecutor's office, probably with other criminal 
justice agencies, and possibly with the county or state data processing 
center. 

A recent Search Group, Inc. research report observed that criminal 
' justice information systems have the potential advantage of avoiding duplica- 
tion of effort by the several agencies involved in the criminal justice 
process.® The same report asserts, however, that some questions of propriety 
are raised by the mere fact that courts are using the same data base as police 
and prosecutors. In these instances, care must be taken to partition data 
sets so that each agency is assured of data privacy and can access only the 
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information to which it is entitled. The main point is that avoiding dupli- 
cation of effort, particularly in data recording, avoids errors and therefore 
provides users with better quality data. . 

Applicable Survey Data . Prosecutors who reported that they use 
or plan to use PMIS were asked whether access is provided to other criminal * 
justice agencies. Responses indicate that 80% of the systems are shared, 
primarily among prosecutors, courts and police. However, it was evident in 
site visits that the degree of data sharing varies. For example, in one site 
the court receives copies of certain PMIS reports but has its own separate 
court information system. In another site, both the prosecutor and court are 
tied into the same data base and system, each having designated data input and 
access. 

Judgmental Assessment . Formal organization structures may not be 
as important as the informal organization structures and interpersonal rela- 
tionships that evolve. Three PMIS projects were observed where the prose- 
cutor was unhappy with the support from the central data processing facility 
(run by a central county data processing agency). Two of these prosecutors 
were not satisfied with the degree of confidentiality of office records in a 
system not under his control. In these situations, characterized by scarce 
resources and inconsistent priorities between user agencies, prosecutors 
desired their own computer and data processing staff. Such an approach 
usually requires a lower cost computer operation — a minicomputer or purchase 
of outside services. The personalities of the prime movers of the PMIS 
project, be they prosecutors, judges or city/county data processing adminis- 
trators, strongly influence the level of interagency cooperation achieved. 

• Extensive data sharing among criminal justice agencies was observed 
in 11 of the 17 sites surveyed. Individual agencies had primary responsi- 
bility for data collection and updating of the PMIS data base at these 11 
sites. Those jurisdictions where extensive data sharing was observed have 
a higher level of interagency cooperation and mutual trust than was observed 
in the limited data sharing jurisdictions. This was true even in non data 
processing activities. Users in extensive data sharing systems have a higher 
level of satisfaction with their systems, particularly when compared to 
jurisdictions having large-scale systems with limited data sharing. Projects 
characterized by extensive data sharing avoid duplication of effort in data 
collection and processing, which results in budgetary savings. In these 11 
jurisdictions where agencies assumed responsibility for updating specific 
items of data consistent with their functional responsibilities, the accuracy 
of the shared data was better than in jurisdictions with duplicate data 
collection procedures. Safeguards on the confidentiality of court/prosecu- 
tion records seemed to be satisfactory. 

If interagency relations permit it, there are significant advantages 
to an integrated criminal justice or prosecutor/court information systems 
approach. These advantages include budgetary savings, data reliability, and 
more effective' interagency coordination of criminal justice activities. As 
lower cost and ever more powerful minicomputer based systems become more 
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available, it may become economically feasible for more prosecutors to estab- 
lish computer systems and supporting staffs of their own. However, even with 
minicomputers, staff support costs remain substantial; moreover, the other 
advantages of the integrated system are not realized under this structure. 

2. Issues concerning the development and operation of PMIS. / 

a. ISSUE : To what extent have prosecutors' management information 
requirements and alternative systems approaches been analyzed and defined 
prior to designing the PMIS? 

How does a prosecutor determine which functions should be included 
in a PMIS and what kinds of information are needed? Organizations that have 
been most successful in developing and implementing a useful management 
information system have started with a thorough analysis of their requirements. 
Of the 17 sites surveyed, a thorough analysis and documentation of require- 
ments was noted in only four non-PROMIS and one PROMIS site. Users who make 
the mistake of first deciding to get a computer or deciding to acquire a 
particular software package before they have defined their own requirements 
and resource constraints are likely to become disillusioned with the burdens 
of making the software work and modifying it to meet their own needs. An 
office could survey the applications that are successful in other jurisdic- 
tions. An outside consultant who is familiar with the literature, state-of- 
art, and relevant experience in the field .could also be helpful. 

Many publications provide expert opinion on the type of data needed 
by prosecutors. Standards have been developed for case management needs. 9 
Information needs for caseflow management, docketing and recordkeeping for 
trial court information systems have been documented by other studies.®*^ 

Various models exist which include PMIS applications that have been 
found useful in other jurisdictions. For example, one widely circulated repqrt 
lists various uses of a PMIS by prosecutors.^ 

After the requirements analysis and preparation of the system con- 
cepts, the prosecutor's office is ready to consider alternative hardware and 
software systems approaches. The most convenient and economical approach is 
to use existing city/county or state computing facilities and data process- 
ing systems staff, as was the case in 15 of the 17 jurisdictions surveyed. 

There are potential problems in such an arrangement, but many jurisdictions 
(12 or 15 surveyed) have been successful in such arrangements; the success 
of a system does not depend on ownership of the computer.^ 

The use of minicomputers^ > 1 1 (such as in Boston and Oakland) and 
microcomputers^ may make prosecutor-owned computer systems more feasible; 
however, the office should still weigh the potential advantages of sharing 
computer facilities and software development and maintenance costs at least 
with the court and other criminal justice agencies, as identified under the 
previously discussed issue on project organization. 
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Judgmental Assessment . In all but five jurisdictions visited, it 
was difficult to obtain documentation or other evidence that a systematic 
analysis of prosecution information requirements and a feasibility study had 
been prepared pr.ior to proceeding with the development of the PMIS. Few alter- 
natives, if any, to the eventual PMIS approach had been examined. Much of this 
can be attributed to the embryonic state of PMIS applications and the lack of ✓ 
data processing systems expertise in most prosecutors' offices. Most projects 
appeared to evolve from a combination of inadequate manpower to cope with large 
caseloads, frustration with court delay and logistical problems, a general idea 
that computers are synonymous with modernization and improved control, sales 
pitches by vendors or consultants, and the availability of Federal funding to 
support the development of PMIS projects. As a result' of all of the above, 
unrealistic expectations and unsatisfactory cost-effectiveness characterize many 
projects. 

On the other hand, those PMIS projects that have been based on sys- 
tematic requirements analyses have evolved in phases, adding new applications 
in accordance with a conceptual design and master plan. In these cases, 
expectations have been realistic and have been met. The adaptation or 
tailoring of concepts and/or software transferred from other sites has also 
been accomplished more effectively when the recipient site had a well-defined 
statement of its requirements. 



b. ISSUE : To what extent have prosecutors transferred PMIS's ope r- 

ating in other jurisdictions, and what advantages have been observed? 



Various prosecution/court MIS or criminal justice information systems 
exist which have the potential for satisfying a prosecutor's needs. One series 
of software system packages, PROMIS, has been transferred to many types of 
jurisdictions and hardware configurations.^ 



There are potential disadvantages to transfer. One is the possibil- 
ity of wasting a great deal of time and money finding out that the system 
being transferred cannot perform the required functions at the new location. 
Most jurisdictions differ markedly in legal procedures, organizations, staff 
capacity, hardware environmment , management style, etc. This means that sub- 
stantial modifications must be made in transferring a system. One advantage " 
in developing a system from scratch is that all the capabilities are specifi- 
cally developed for that jurisdiction. Additionally, the prosecutor and his 
staff develop pride of ownership (and resultant enthusiastic use), and they 
are involved in the learning curve from the beginning. Two jurisdictions 
(Chicago's Automated Case Management and Oklahoma's ADRS) report a low cost of 
development from scratch of a prosecution system.6»15 g n the other hand, 
such development can be a costly experience if one does not have the proper 
technical staff available. 



Nevertheless, transfer can make good sense not only because of the 
potential savings in time and cost, but 'because of the uniformity it provides 
for consideration of information requirements. - System transfer has been 
encouraged by LEAA' funding policies. New LEAA funding policies related to 



>» 




40 



f 





prosecution or court information systems provide grants for statewide PROMIS 
under LEAA's "Incentive Fund Program." This policy encourages multi-jurisdic- 
tional implementation of PROMIS in prosecutors' offices and trial courts. 16 

Applicable Survey Data . Fifty (about 57%) larger prosecution 
offices (over 25 employees) with operational PMIS projects or planning a PMIS / 
indicated PROMIS as their type of system. This suggests a very high interest 
in technology transfer. 

Most prosecutors who undertake PMIS transfer projects do so to save 
time and money. They can also assess their needs better when they can 
observe a well defined model in operation at another jurisdiction. 

Judgmental Assessment . As discussed later in Sections II E.4.b and 
c, little evidence is available from this study to support the notion that 
transfer of PMIS systems has resulted in lower development costs. Unfortu- 
nately, with the current state of the art, many transfer projects have required 
substantial investments in modifying software to fit different hardware/soft- 
ware environments, as well as different organizations, and local rules and 
procedures. Under these circumstances, it can indeed be more costly to invest 
the time and effort to study, test, . evaluate, modify and debug an existing 
package. 

The state-of-the-art in software systems packaging, modular design, 
data base management systems, tailoring facilities, pre-compilers, and pararrr 
eterized report generators has been advancing so rapidly that the picture on 
economics of technology transfer may be undergoing a significant change. Even 
the approach to technology transfer needs a thorough review. It may not make 
sense to think of transferring an entire system, only certain modules, appli- 
cation programs, or software facilities. One should expect to do some special 
programming or modification; the issue is to minimize the extent of it. For 
example, San Bernardino has developed a single interface control program which 
•communicates with all application programs. In any transfer to a different 
hardware/software environment, only the interface control program would need 
to be modified. The application programs are all written in a highly transfer- 
able ANSI COBOL. More flexible vendor-supplied software packages, including 
tailoring, report generation, data base management, generalized inquiry and 
statistical packages will also have an impact on technology transfer strategies. 



c. ISSUE : To what degree has external funding support and technical 

assistance been necessary in PMIS projects? 



Prosecutors generally have very limited financial resources. Is it 
therefore wise to seek external funding, at least for the development of the 
system? The dual problem of obtaining both the funds to develop and install a 
court information system, and to continue operating the system (i.e., institu- 
tionalizing it) was raised in one research report >7 and is a question all 
prosecutors planning as PMIS must confront. 
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Another recent evaluation found that apparently little thought was 
given to local funding of the criminal justice information system after Federal 
funding ran out. 1 8 City police agencies were among the users that expended 
more effort as data providers than as data users; hence, they did not feel they 
were benefiting enough to contribute to the cost of operation. This evaluation 
recommended providing management information reports (e.g., arrest statistics 
by arresting officer's name) in return for data input by the police. Agencies 
which were equally data users and data providers were quite willing to absorb 
a share of operating costs. 



Prosecutors also generally have very limited staff resources. To 
what extent should they attempt to use their own personnel to design and 
develop a PMIS? To what extent should they use outside consultants? A recent 
study reports that both extremes ("Turnkey" or total outside consultant, and 
"Total In-House") possess inherent problems and disadvantages which outweigh 
any possible advantage of either approach. 2 The same study concludes that a 
combination of consultant support with professional in-house staffing is the 
proper solution and recommends outside consultants conduct the initial feasi- 
bility study. 

Applicable Survey Data . Approximately 46 percent of the offices 
using a PMIS reported that a contractor was involved in development of the 
system. Offices that reported using a PMIS were queried on the source of 
funding for their projects. Responses indicate that external funding support 
was necessary in over 80 percent of the 40 operational PMIS projects responding. 
The significance of LEAA for development of PMIS's is shown by the data 
indicating that two-thirds of the projects relied on LEAA funding support. 

• Judgmental Assessment . None of the sites visited would have under- 
taken their PMIS project without LEAA funding support. Indeed, due to limited 
budgets and staffing, and a natural resistance to automation on the part of 
many attorneys, the rate of adoption of automated PMIS projects would probably 
have lagged considerably had LEAA support not been available. Only the recent 
advances in lower cost computer technology would otherwise have stimulated 
interest. 

The need for LEAA funding is usually associated with technical assist- 
ance which the majority of projects utilized. Of the 17 PMIS sites visited, 
ten used an outside contractor, in varying degrees, to advise and/or carry out 
software development. Outside contractors have been used most effectively 
where there is an in-house systems capacity present to properly direct the 
effort and utilize the results. 

d. ISSUE : Have constraints, such as privacy/security safeguards and 

budgets, inhibited the development and operation of PMIS projects? 

What are the appropriate controls for safeguarding security and 
privacy, and to what extent have they been implemented in the existing systems? 
Federal regulations pertaining to privacy focus on "criminal history record 
information" compiled by criminal justice agencies on criminal offenders. 
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This information is recorded as the result of arrest, detention, or other 
initiation of criminal proceedings. It does not include intelligence, analyt- 
ical and investigative reports, or statistical records and reports in which 
individuals are not identified and from which personal identifiers are not 
ascertainable. The rules and regulations provide specific exceptions for 
certain original records of entry, court dockets, and court records of public' 
proceedings. 20 Although these exceptions appear to exempt the courts, court 
managers have been cautioned that the exemptions apparently relate to manual 
records and case files not to automated files created therefrom. This satis- 
fies the definition of a criminal history record. ? 

Most state and Federal regulations have failed to address the issue 
of when a current court/prosecution record becomes a criminal history record. 
PMIS defendant records with no current cases pending may be purged and placed 
on a criminal history file.H At such time, they may come under the criminal 
history regulations. 

One purpose of the privacy regulations was to eliminate the use of 
FBI rap sheets (criminal histories) which did not record the court disposi- 
tions of arrests. Dispositions must be entered into the files within 90 days 
and most PMIS's provide this capability. Generally, dissemination of automated 
information is restricted to criminal justice agencies. 

There are statutory provisions for sealing or expungement, physical 
security, individual access, and review (including defendants and incarcer- 
ated persons) .19,21 The PMIS should provide logging of all access requests, 
terminal I.D., password controls, physical control, backup, and other tech- 
niques designed to safeguard privacy and confidentiality of records. 15,22 

Applicable Survey Data . Reasons for not using or planning to use 
a PMIS were solicited and the results reveal that budget and staff are the 
most important constraints for both larger and smaller offices. Next in 
importance was the opinion that there was no need for a PMIS. This opinion 
often appeared to coexist with the budgetary and staffing factors. Privacy 
was considered the least important constraint. 

Judgmental Assessment . Privacy and security constraints have had 
substantially less of an inhibiting effect than budgetary and staffing 
limitations on PMIS development and operation. Nevertheless, in two sites 
visited, there was concern expressed by the prosecutor about safeguarding the 
confidentiality of his records, particularly when the. PMIS operation was 
outside of his control. The political nature of the district attorney's 
position makes the distribution of statistical measures of performance highly 
sensitive in many jurisdictions. 

In all sites visited, prosecutors perceive very tight budgetary and 
staffing constraints on their operation. Many of those not perceiving a need 
for a PMIS believe that they have neither the staff nor budgets to even 
consider it. Even those who do plan or use a PMIS would have been unlikely 
to do so had not external funding support and technical assistance been 
available. 
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e. ISSUE : Are PMIS computer programs and procedures being adequately 

documented so that the system can be maintained and improved, and technology 
transfers accomplished? ~ — - " 

An important factor in assessing PMIS projects is the extent and 
quality of the available documentation. Sound documentation is crucial to 
proper maintenance of the system, particularly in the event of personnel ' 
turnover, as well as system transfer. LEAA has observed that two of the 
largest obstacles to be overcome, once the decision has been made to trans- 
fer a system, are inadequate documentation and the fact that certain computer 
languages restrict themselves to one computer manufacturer. 23 Hence, LEAA 
developed a set of special conditions for grants involving automated data 
processing. Among other conditions, applicants must agree: 

• To use, to the maximum extent practical, computer soft- 
ware already produced and available without obligation. 

To assist in this area, LEAA publications and regional 
systems specialists should be consulted. 

• That all application programs be written in Federal 
Standard COBOL or ANSI FORTRAN whenever possible. Pro- 
grams may be written in ANSI BASIC for microcomputers 
and minicomputers, subject to certain conditions speci- 
fied by LEAA. 

• To provide complete documentation in accordance with 
Federal Information Processing Standards (FIPS Pub. 38). 

Applicable Survey Data . Almost 90 percent of offices responding 
to the mail survey indicated they had systems flow, data base and output 
reporting documentation; fewer (74%) had documented their programs. In 11 of 
the 17 surveyed projects, documentation was available in various forms. How- 
ever, only five non-PROMIS sites had finalized and organized their documenta- 
tion in a fashion suitable for technology transfer. 

Judgmental Assessment . Most PMIS projects have tried to document the 
various components of their system. Many programmers, however, make documenta- 
tion the final task and sometimes never get around to completing it because of 
other crises that develop. Because of the challenge in writing separate tech- 
nical and non-technical manuals for different audiences, a complete set of 
well organized documentation, prepared in several volumes to address management, 
systems/programmer personnel, data entry personnel, operators and user needs, 
is rare. 

PMIS project managers desire to complete their documentation, but 
usually run out of external funding support before documentation is completed. 

As a group, PROMIS projects have above average documentation supplied by 
INSLAW under the LEAA-funded PROMIS transfer program. However, many projects 
have made substantial modifications and these are often not well documented. 
INSLAW has, however, designed some, self documenting facilities into its latest 
version, MAXI-PROMIS. 
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Issues Concerning the Impact of a PMIS on the Criminal Justice 



a. ISSUE: Have PMIS projects specified their goals and intended 

capabilities, and~to what extent have these goals been met? 

By the effective use of information, prosecutors should be able to 
meet goals they have set for improved office performance. Before one can 
attempt to assess impact, it is important to assess whether PMIS's are pro- 
viding the capabilities that they set out to provide. A recent evaluation of 
a multi-agency criminal justice information system found that managers were 
expecting much more than they were receiving. Many felt the system only 
supported clerical functions.^ City officials thought the system primarily 
supported county functions; county officials viewed the criminal justice 
system as primarily a supporter of police functions of cities. The evaluator 
concluded that this was symptomatic of early project over-selling of the 
intended capabilities of the system. 

Applicable Survey Data . Prosecution offices were asked to assess 
the contribution that an ideal PMIS could make to the attainment of certain 
office goals. Those offices that actually had a PMIS were asked to rate the 
expected and actual contributions of their system. 

Table 5 summarizes the comparative contributions of the ideal PMIS 
to the expected and actual contributions of their own PMIS projects. The 
first column shows, on a scale of 0 to 100, an index of the contribution that 
responding prosecutors, on average, believe an ideal PMIS could make to 
attaining office goals. Scheduling and logistical control are perceived as 
the highest area of payoff and an increased conviction rate as the lowest area 
of payoff. Comparison of expected and actual contributions of their own PMIS 
projects reveals a gap, particularly for attaining evenhandedness. On the 
other hand, the gap is smaller for the scheduling goal. 

A large gap between actual and expected performance can lead to a 
sense of frustration on the part of prosecutors. Table 6 summarizes the 
ratio of the perceived actual contribution to the expected contribution as 
well as the ratios between the actual and ideal contributions. The ratio of 
actual to expected contributions indicates that PMIS projects fall short of 
prosecutors' expectations. This is true for both PROMIS and non-PROMIS 
projects. PROMIS projects fare better than non-PROMIS projects in contrib- 
uting to the goal of priority allocations of staff; they are less helpful in 
scheduling and logistical control and improving the conviction rate. On the 
ratio of expected to ideal PMIS contribution, PROMIS projects score consist- 
ently higher than non-PROMIS, which suggests that PROMIS users continue to 
have relatively high expectations for their projects. 
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Table 5. Comparison of Ideal to Expected/Actual Contribution 

of Prosecutor MIS to Management Goals - Jurisdictions 
With 25 or More Employees (median percentage contribution) 



Prosecutor Goals 


Contribution of Ideal 
MIS to Management" Goals 
N = 75 ' 


Expected MIS 
Contribution 
N = 73 


* 

Actual MIS 
Contribution 
N = 30 


Allocate staff 
by priorities 


79 


70 


33 


Monitor even- 
handedness 


, ' 80 


70 


12 


Control schedules 
and logistics 


89 


80 


73 


Perform research 
and analysis 


81 


74 


33 


Evaluate office/ 
prosecutor 
performance 


75 


60 


24 


Increase con- 
viction rate 


49 


30 


10 

i 



Source: PMIS Assessment Mail Survey, Westat, Inc., July 1979 

NOTE: N's vary slightly for each goal. N's for the "ideal" and "expected" 
columns include jurisdictions operating and planning a PMIS; the "actual" 
column represents only operational PMIS projects. 

Given the ratios shown in Table 6, one would like to combine them 
into a single composite index that could be used in making an overall assess- 
ment of project types. A simple mean of the ratios across all the goals would 
amount to giving each goal equal weight. An alternative, used in this analysis, 
is to weight the ratio of actual to expected for each goal by the relative impor- 
tance weight given that goal, and then computing an average across all goals. 

Two methods of computing the relative importance weights were used and are dis- 
played in Table 6. The first, composite index A, uses each individual respon- 
dent's own goal importance rating as a weight; the second, composite index B, 
uses the mean of the goal importance weights given by all respondents as a weight 
for each goal. The ratio for each goal is then multiplied by the appropriate 
goal importance weight and averaged across all goals .to compute each respective 
composite index. On the basis of the individualized goal importance weights 
(Composite Index A), there is almost no difference between PROMIS and non-PROMIS 
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Table 6. Perceived PMIS Contribution to Goal Attainment, 
Jurisdictions With 25 or More Employees 



r 


Prosecutor Goals 


Ratio of 
Actual/Expected 
Contribution 


Ratio of 

Actual/Ideal 1 

Contribution 


Ratio of ' 

Expected/Ideal 
Contribution 


r 


Non- 

PRGWT3' 

(N=20) 


PROMIS 

(N=8) 


Non- 

PROMIS 

(N=20) 


PROMIS 

(N=8) 


Non- 

PROMIS 

(N=24) 


PROMIS 

(N=12) 




Allocate staff 
by priorities 


.54 


.64 


.39 


.58 


.74 


.90 




Monitor even- 
handedness 


.44 


.45 


.32 


.38 


.71 

• » 


.82 




Control schedules 
and logistics 


.84 


.61 


.73 


.56 


.86 


.87 


€ 


Perform research 
and analysis 


.61 


.55 


.49 


.53 


.85 


.94 




Evaluate office/prose- 
cutor performance 


.56 


.51 


.40 


.44 


.76 


.87 


e 


Increase conviction 
rate 


.61 


.48 


.43 


.39 


.69 


.82 


& 


Composite Index A 
(Individualized goal 
weights) 


.45 

(11) 


.44 

(6) 


.37 
(13) • 


.40 

(6) 


.58 

(15) 


.65 

(11) 


ft 


Composite Index B 
(group mean goal 
weights) 


.43 

(14) 


.34 

(7) 


.33 

(17) 


.31 

(7) 


.53 

(19) 


.58 

(12) 



Source: PMIS Assessment Mail Survey, Westat, Inc., July 1979 

Note: The N's for non-PROMIS sites varied slightly for each goal. 
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projects in the ratio of actual to expected contributions. Using Composite 
Index B, the non-PROMIS projects have a higher composite index of actual to 

expected contribution. The individualized weights result in a somewhat higher j 

composite index for PROMIS than for non-PROMIS with respect to the actual to 
ideal ratio. Again, weighting each goal the same across the entire population 
results in a less favorable position for PROMIS projects. This suggests that, 

PROMIS users have somewhat different goals and expectations than non-PROMIS 

users, and that their actual projects are more in line with their own goals 1 

and expectations than with the group average. 

One variable that should be controlled in assessing the perceived 
gap between actual, expected and ideal PMIS contributions to office goals is 
the length of time that a sysfeem.-thas been operational. Normally, one would 
expect the ratio of actual to ideal to increase over time, as "bugs" are 
eliminated from the system and capabilities are tailored to office needs. 

Table 7 presents analyses of the ratios of actual to expected, actual 
to ideal and expected to ideal PMIS contributions, controlling for length of 
time the PMIS has been operational. The composite index was computed using 
the individualized goal importance weights defined earlier. 

PROMIS users score higher than non-PROMIS users in the ratio of 
expected to ideal PMIS contribution; i.e., they expect their projects to 
approach the ideal PMIS. This is consistent with the earlier finding for 
Table 6. However, when one compares the ratios of actual to expected or 
actual to ideal, the non-PROMIS projects score higher for most goals than the 
PROMIS projects. 

Duplication of effort was observed in eight of the 17 jurisdictions 
surveyed. The degree of duplication, however, varied widely: in two of the 

eight projects (both non-PROMIS sites), card files containing extensive 
duplicative data were being maintained as backup to the PMIS; in two other 
jurisdictions (one PROMIS, one a non-PROMIS site), separate forms (source 
documents) were being prepared for PMIS data input operations, thus data were 
being recorded three times (original report, source document, and entry into 
PMIS); and, in the other four jurisdictions (one non-PROMIS and three PROMIS 
sites), statistics were being kept manually because users were not satisfied 
with the accuracy of the data. 

Judgmental Assessment . A written statement of PMIS goals and 
requirements had been prepared in only five surveyed projects. The prosecu- 
tors' expectations of PMIS capabilities usually came from contacts with other 
jurisdictions, vendors,, outside consultants, magazine articles and profes- 
sional association meetings. A common method by which prosecutors learn of 
PMIS capabilities is through the exchange of information with other prosecu- 
tors. Even if prosecutors do not specify their goals in advance, they 
do have certain expectations of their PMIS projects. j 

i 

. , : J 

The most common applications of PMIS projects are on-line defendant i (! 

and case tracking or status reporting, workload reports and calendar displays. M 
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Table 7. 



Prosecutor Goals 



Allocate staff 
by priorities 

Monitor even- 
handedness 

Control schedules 
and logistics 

Perform research 
and analysis 

Evaluate office/ 
prosecutor 
performance 

Increase con- 
viction rate 



Composite Index 



Preceptions of PMIS Contribution 
to Prosecutor Goal Attainment for 
Projects Operational 1-3 years - Juris- 
dictions with 25 or More Employees 



Ratio of 
Actual to 
Expected 


Ratio of 
Actual to 
Ideal 


Ratio of 
Expected to 
Ideal 


Non- 

PROMIS 


PROMIS 


Non- 

PROMIS 


PROMIS 


Non- 

PROMIS 


PROMIS 


7 


7 


7 


7 


8 


8 


.66 


.58 


,58 


.54 


.85 


.94 


.49 


.42 


.42 


.35 


.84 


.79 


.82 


.60 


.77 


.56 


.95 


.90 


.58 


.59 


.53 


.58 . 


.93 


.98 


.72 


.48 


.61 


.43 


.89 


.92 


.67 


.39 


.52 


.35 


.84 


.85 


.52 


.44 


.44 


.40 


.63 


.71 



In one jurisdiction, prosecutors have effectively used on-line facilities to 
prepare one case by examining data across several cases for particular 
defendants and witnesses. This information can be used in plea bargaining or 
in discrediting witnesses with prior records. ' In another jurisdiction, judges 
have used on-line facilities to check defendant or attorney supplied informa- 
tion in support of continuance motions. 

As prosecutors and their systems staff obtain more experience in 
using their PMIS's and exchange information, an increasing number of innova- 
tive and more sophisticated applications can be anticipated. For example, 
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Alameda County's DALITE and Los Angeles' PROMIS projects have been attempting 
to use case weighting approaches to assist in .determining staff needs. The 
weights reflect the staff requirements of different case types. Several 
informative research designs and findings have been reported by INSLAW using 
the PROMIS data base of the U. S. Attorney's office of Washington, D. C., as 
well as a cross-city analysis of other PROMIS projects. Oklahoma's ADRS , 
project has a sophisticated statistical analysis and reporting capability 
which can serve as a model for management analysis applications. Other exam- * 

pies of innovative PMIS applications can be seen: in Tarrant County, Texas, 

where the prose for various charges are stored on computer files so that 
charging documents can be rapidly generated whenever charges are changed 
(e.g., by Grand Jury action); and, in Norfolk, Virginia, where arrest reports 
and arrest warrants are automatically generated using on-line booking opera- 
tions. 

Analysis of the mail survey responses and site visits indicate a 
general shortfall in the degree to which PMIS projects have met prosecutor's 
goals. The situation is better with respect to scheduling and logistical 
control applications than it is with research and analysis applications. The 
gap between expected and actual goal attainment will narrow as prosecutors 
gain more experience in using their PMIS. Implementation of new applications 
will help the situation, but it is also anticipated that prosecutor's expecta- 
tions will become more realistic with PMIS experience. 

b. 1 5SUE : Have PMIS projects been useful in generating performance 

measures, and are they used to improve management and operations? 

PMIS introduces a capability for performance evaluation that did 
not exist previously. Even though PMIS information is a tool of the prosecutor, 
the existence of information which can be used for evaluation purposes must 
introduce an awareness of performance evaluability which is a new factor in 
prosecution behavior. What is the effect of this new awareness? Is it to make 
the prosecutor more cautious? Does it motivate the prosecutor to adopt partic- 
ular measurable goals and to try to devise strategies and tactics to implement 
these? Does it really motivate and enable more effective prosecution management 
practices? 

More research than ever before is being devoted to- improving measures 
of prosecution performance .and the quality of justice. A performance measure- 
ment program has been proposed and a preliminary analysis has been made of the 
usefulness of PMIS's in satisfying those requirements. 24 Recent awards have 
been made by the National Institute for Law Enforcement and Criminal Justice 
to study the area of performance measurement for prosecutors and for courts. 

Various research reports have been published related to the subject of prose- 
cution performance measurment, based on data generated by a PMIS. 25 

One problem in developing and interpreting performance measures is 
the wide variation in procedures and terminology among local jurisdictions. I 

It could be very illuminating to compare various jurisdictions with respect 
to uniformly-defined performance measures.. For example, Church, et al. , 
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recently defined uniform measures of court case processing time in a study of 
court delay. 26 By taking a systematic sample of felony cases terminated in 
21 courts of general jurisdiction, they were able to develop cross-comparable 
measures of case processing time and certain court characteristics. The 
potential advantages of having a number of jurisdictions routinely collect a 
set of uniformly-defined data are very attractive as demonstrated by Brosi in' 
a recently released research report. 27 

Applicable Survey Data . The mail/telephone survey examined the 
availability of statistical data for possible evaluation purposes. Table 8 
summarizes those responses. For the types of data specified, PROMIS users 
appear to perceive a greater availability of statistics than non-PROMIS 
users. Among users, case load statistics are more available than either 
delay or disposition statistics. 




Table 8. Percentage of Sites Reporting Data 
Available from MIS - Jurisdictions 
With 25 or More Employees 
(N's in parentheses) 







f 



Judgmental Assessment . A majority of the PMIS projects report that 
they generate measures related to workload, court delay, and disposition rates 
of prosecution/court performance. In most of the sites visited where such 
information was available, the prosecution managers were using the statistics 
to manage their workloads, e.g. , assigning cases to individual prosecutors, 
prodding judges to move cases faster, and developing office plea bargaining 
strategies. PMIS performance measures have influenced policy development 
and decisionmaking as in the establishment of witness assistance/notification 
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units, career criminal programs, joint prosecution/court projects to reduce 
court delay, special projects aimed at specific types of crime and development 
of uniform charging policies and procedures. 



c. . ISSUE : How cost-effective have various types of PMIS projects 

been in collecting, processing and displaying information? 



Cost-benefit analyses have proved, effective in justifying prosecu- 
tion management information systems. 28 An extensive set of projections can 
be computed on the cost reductions expected through such capabilities as 
automated subpoenas and more efficient use of personnel and/or eguipment. 



It has been reported that court personnel often have negative reac- 
tions to the introduction of computer systems. 29 There are, however, certain 
intangible benefits which are difficult to measure, including: 



• Improvement in the adjudication process through more 
effective, efficient and timely operations; 



• Improved public image resulting, hopefully, in improved 
cooperation; and 



• Improved operations and management through research 
results made possible by the availability of data. 7 



Case scheduling and logistical control capabilities are useful for 
both prosecution and court management. Capabilities of the PMIS are also of 
potential use to prosecutors for research, performance evaluation capabili- 
ties and monitoring evenhandedness in prosecutory and judicial decisions. 

The ability to track cases and defendants from arrest through final court dis- 
position is important to police, prosecution, court and correctional agencies. 
Nevertheless, while PMIS projects prove useful to prosecutors, are they worth 
the costs in staffing and computer time? Are certain types of projects more 
cost-effective than others? 



Applicable Survey Data . Prosecution offices that reported having 
a PMIS were asked to give their development and operation costs. Among 24 
responding jurisdictions with 25 or more employees, the median development cost 
was $150,573. The median monthly operating cost was $3,500. 

The number of offices that have a PMIS and could provide cost data 
in responding to the mail survey was relatively small, making any inferences 
from the sample to the population problematic. Given that caution, Table 9 
reports the average development and monthly operating costs per case screened 
and per case prosecuted in 24 offices that responded to both cost and workload 
questions. The table compares PROMIS and non-PROMIS projects to the extent 
that data were available from the mail survey. All else being equal, one 
would expect projects such as PROMIS, which involve systems transfer, to 





• 52 



I 



I 



show lower development costs than other PMIS projects. However, there is no 
support for this theory in the data provided by this limited sample of PMIS 
projects. 



Table 9. Development and Operating Cost 

Jurisdictions With 25 or More Employees 



TYPE OF COST 



TOTAL 



DEVELOPMENT 



- Median 



- Mean cost per 
case screened 

- Mean cost per 
case accepted 

MONTHLY OPERATIONS 

- Median 



Mean cost per 
case screened 

Mean cost per 
case accepted 



$150,573 

(24) 

$ 33.51 

( 10 ) 

$ 41.82 
(14) 



$ 3,500 
(24) 

$ .37 

( 11 ) 

$ .50 

(16) 



Source: PMIS Assessment Mail Survey, Westat, Inc., July 1979 



Judgmental Assessment . The data gathered in Phase I give some indi- 
cation of patterns associated with apparently effective PMIS projects, as dis- 
tinguished from apparently ineffective ones. Some projects were more effective 
in certain aspects of data processing than other aspects; e.g. , some projects 
had apparently effective operational support applications, but inadequate 
management information generation facilities. ' 

Certain characteristics were observed in PMIS projects with apparently 
cost-effective data processing operations. One of the most effective project 
situations was one characterized by extensive data sharing either in an inte- 
grated prosecution/court MIS or an integrated criminal justice information 
system, supported by effective large-scale central data processing facilities. 
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In most of the effective projects, the prosecutor relied principally on court 
supplied data with the prosecution staff supplying data on attorney assign- 
ments and prosecution witnesses. Assigning the data entry role to the single 
agency responsible for originating the particular data has systematic advan- 
tages in achieving greater reliability and reduced costs (by avoiding dupli- 
cation of data collection effort). 

Another type of project with apparently high cost-effectiveness 
was one based on the use of a minicomputer; under the prosecutor's control. 

To be effective, these projects relied on a few experienced data entry clerks 
who were capable of either directly recording activities in court or coding 
data from the prosecutor's case file. 

A third type of project that was apparently cost-effective was a 
large-scale system that served multiple jurisdictions with remote on-line 
terminals or remote batch data entry facilities for participating agencies. 

The various jurisdictions received apparent economies of scale by sharing a 
competent central systems staff and facilities. 

On the other hand, some types of projects surveyed were not per- 
ceived as cost-effective. One class of such projects is characterized by 
limited data sharing, prosecutor control of data entry, use by a single 
jurisdiction, and use of large-scale computer facilities operated by a 
central city or county data processing unit. In some situations, problems 
were caused by a lack of interagency cooperation and conflicting priorities. 

During the site visits, it appeared that sites with limited data 
sharing also had limited interagency cooperation in areas other than data 
processing, while the opposite was observed in sites with extensive data 
sharing. The cost-effectiveness of these projects suffered from unresponsive 
central data processing support for needed system improvements or modifica- 
tions, more burdensome data collection procedures than expected, questionable 
data accuracy, and higher than expected operating costs. 

Another ineffective class of PMIS projects was one characterized by 
a large-scale, ambitious integrated criminal justice information system but 
without either the necessary interagency cooperation and leadership or the 
required central data processing support capacity. 

The 17 PMIS projects surveyed have been effective in supporting 
day-to-day operational data processing needs. Four of these systems, however, 
have not satisfied prosecution desires for information on such subjects as 
crime specific performance measures and disposition rates, but such capabil- 
ities have been planned as a next step in a logical evolution of increasing 
capacity. Thirteen projects have produced satisfactory management reports. 
However, special problems or crises often arose creating the demand for an ad 
hoc research inquiry. Those projects that had the most effective management 
information reporting capacity usually had generalized and flexible statistical 
and management inquiry software. In two jurisdictions with recent turnover in 
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prosecution management personnel, prosecution managers were not aware of the 
types of PMI5 information that would be useful to them. In two other juris- 
dictions, the prosecutors were well aware of these needs but were frustrated 
by the delay in getting such information. 

d. ISSUE : To what extent can the cost, benefits, and impact of t 

PMIS projects be objectively evaluated? 

An in-depth evaluation of the impact of a PMIS project must describe 
the project's objectives, activities, intended effects, and the changes that 
occur in the environment. In addition, it should assess whether changes result 
from the PMIS or from other intervening events. 

In their publication, "Intensive Evaluation for Criminal Justice 
Planning Agencies," Weidman, et al. present various designs for conducting an 
in-depth evaluation. 30 They state that many evaluations fail to link program 
activities and results, or definitively describe what actually occurred after 
funds were expended. The collection of information for intensive evaluation of 
prosecutor management information systems may be difficult because much of the 
necessary data may not be available or may be very costly to obtain. 

The availability of baseline data is one consideration in determining 
whether a project's impact can be properly evaluated. Phase I of the NEP is 
designed to document the chain of assumptions linking the expenditure of funds 
to project intervention, the project activities to immediate outcomes, and the 
immediate outcomes to the impact on the problem addressed by the topic area. 

Applicable Survey Data . Descriptive statistics have to be used with 
care. They should be verified and any analysis should be carefully controlled 
to avoid spurious conclusions due to ignoring certain variables. To determine 
the reliability of the responses to the mail survey, a question was asked on 
whether or not the data came from records. The responses are summarized in 
Table 10 and indicate that caseload data are apparently more available than 
delay data and that some delay statistics are more available than others. 

There is a clear pattern that those jurisdictions using an MIS were 
able to supply more delay data from records (43.7%) than those planning a PMIS 
(19.1%) or those neither planning nor using a PMIS (16.8%). Among offices 
planning a PMIS, prosecutors planning for PROMIS appeared to acquire their 
delay data from records more often (30.3%) than those planning non-PROMIS 
projects (7.9%). 

Judgmental Assessment . The Phase I study of literature and other 
data sources revealed many diverse methods of calculating costs and assessing 
project benefits. These resulted in noncomparable measures among PMIS projects. 
Objective assessments of benefits and impact can be accomplished only if 
uniform definitions of cost elements, performance measures and sound evaluative 
research methodology are applied across projects by a disinterested qualified 
third party. 
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Table 10. Percent Reporting Data Acquired From. Records - 
Jurisdiction With 25 or More Employees 
(N's in parentheses) 



Delay Data 



Using MIS 

Non- 

PROMIS PROMIS 



Arrest To: 

Filing 

Indictment 

Trial verdict 

Misdemeanor guilty plea 



Felony guilty plea 
related to indictment 

Nolle/dismissal 



Case Load Data 
Intake and dispositions 



Source: PMIS Assessment Mail Survey, Westat, Inc., July 1979 



Sufficient data exist to determine development and operating costs 
of PMIS projects. Uniformly defined data collection instruments can be used 
both in extracting data from records and in conducting key informant inter- 
views. Reasonable estimates can be made to fill in gaps. 

Since it is not feasible to evaluate the PMIS project intervention 
within a controlled experimental framework, a quasi-experimental design must be 
used. To attribute changes due to the PMIS intervention, the design should pro- 
vide sufficient observations, and statistical controls for variation in case 
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characteristics, court characteristics, policy and other variables that might 
affect performance simultaneously with the PMIS. This requires an in-depth 
analysis beyond the scope of the Phase I site visits. 

Measurement of PMIS effectiveness and impact is quite different from 
the usual program evaluation situation. PMIS projects have an advantage in 
that they can be self-monitoring; i.e., they generate the prosecution/court 
performance measures that, in turn, can be used to measure PMIS impact. A data 
base can be developed from PMIS case/defendant history files to show changes 
over time in a time series format. 

One problem that occurs in many program evaluations is the unavail- 
ability of reliable base line (conditions existing before the program 

intervention). However, PMIS project evaluations have an advantage over other 
types of project evaluations in the collection of baseline data in that the 
nature of the PMIS project is a chain of events: 

(1) Install PMIS; 

(2) PMIS measures prosecution/court activity; 

(3) PMIS users review statistical reports; 

(4) PMIS users make policy and operational 
decisions; and 

(5) Repeat cycle, starting from (2) above. 

The PMIS-generated measurements of prosecution/ court activity (Step 2) would 
need to precede the use of that information and any impact on prosecution/court 
performance. Thus, the data collected in Step 2 constitute a set of baseline 
data. However, since prosecution/court users must assimilate the technology 
of the PMIS and learn how to use it effectively f the impact of the PMIS is not 
expected to be immediate. 

Some effects of the PMIS project may result from the development 
effort itself and the associated staffing and organization activities. The 
baseline data for analyzing these effects can usually be gathered from key 
informant interviews, and reviews of office records and procedures. Manual 
samplings of court case records also can be conducted to fill in data gaps in 
the PMIS data base and develop baseline data for the period preceding the 
PMIS operation. 

e. ISSUE: Have PMIS's had any measurable impact on the prosecution/ 

court process and criminal justice system? 

A PMIS that provides its intended capabilities may still fall short 
of having any impact on the prosecution/court process. For example, the PMIS 
can enable prosecution management to monitor evenhandedness. Beyond monitoring, 
however, is the attainment of evenhandedness itself. It is possible that 
PMIS, especially in the presence of particular prosecutorial policies, creates 
greater evenhandedness in charging, plea bargaining, trial outcomes, and in 
sentencing. This potential PMIS impact should be systematically investigated 
in any in-depth evaluation. 
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Some published research reports demonstrate that the PMIS can be 
employed to evaluate aspects of police and judicial performance. 25 Recid- 
ivism studies have obvious implications for the corrections process as well 
as for prosecution and sentencing policies. 

Judgmental Assessment . The Phase I effort revealed no instances 
where the impact of an MIS on the prosecution/court process had been ade- ' 
quately measured. Any impact assessments in Phase I are necessarily judgments. 

About one third of the 17 project sites visited appeared to have 
utilized their PMIS's to a sufficient extent that the project might have a 
measurable impact on prosecution or court performance. Fifteen projects 
reported improvements in case scheduling and logistical control; nine projects 
reported reduced backlogs and improved rates of disposition. 

Prosecutors participating in the PROMIS cross-city analyses have 
focused on reducing attrition (losses of cases through declinations, nolles 
and dismissals), or on dropping cases earlier to conserve resources. Some 
prosecutors may be able to achieve increased effectiveness in dealing with 
certain crimes and certain types of offenders, e.g. , career criminals, with 
assistance from their PMIS projects. Through more systematic pse of. their 
PMIS capabilities, some jurisdictions may be able to show an increase in the 
proportion of arrests that end in convictions. 

Only two sites were observed which systematically reviewed measures 
of evenhandedness. One site did compare disposition rates across prosecutors 
to attempt to evaluate individual performance and flag an imbalance in the 
proportions of pleas, trials, nolles and dismissals. 

In two situations, the PMIS was reported to have a negative impact 
on prosecution/court performance by siphoning off scarce manpower to prepare 
computer inputs rather than process cases. These PMIS projects received only 
limited use and could be expected to have no discernible impact. 

4. Issues Concerning the Evaluation of PROMIS 



a. ISSUE : 
non-PROMIS projects? 



Are PROMIS projects more comparable and evaluable than 



One significant advantage expected from PROMIS projects has been that 
the uniform concepts, common terminology, data definitions and other common 
features would make these projects comparable and thus permit cross-city 
comparisons. To the extent that common terminology and procedures were used, 
there might be economies of scale in the availability of technical assistance, 
research and development and education/training programs. 

Applicable Survey Data . During the site visits, the typical site 
generated adequate operational and management statistics on a periodic basis. 
Three non-PROMIS sites had available a general purpose statistical software 
package which could be used in special evaluative analyses. The typical 
PROMIS project has such a statistical software package. 
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Another important dimension in examining the evaluability of. PMIS 
projects is the range of applications in use. These have been recorded for 
all sites visited. 

•In reviewing applications designed to support operational control, 
it was observed that case, charge and defendant tracking applications are 
common for 1 00?o of the PMIS projects surveyed. There is a drop-off to 80% 
in calendar displays and persons tracking capacity (witnesses, victims, 
attorneys, etc.); and witnesses notices are produced by only 41% of the 
surveyed sites. On-line booking is non-existent, of course, among non-inte- 
grated projects, but used in seven out of 10 of the integrated systems. 

A review of applications designed to support management uses of 
the PMIS reveals that case inventories are produced at all 17 survey sites. 
Workload reports are produced at 83% of the sites, while case aging (at 30%) 
and case weighting (at 12%) are much more rare. The latter features consti- 
tute a method of estimating manpower requirements based on case complexity; 
i.e., the case is given a different weight based on the degree of complexity 
or other attributes which affect staffing requirements. PROMIS projects 
score substantially higher (83%) than non-PROMIS projects (30%) in crime- 
specific and attrition analysis capacity. In general, a highet proportion of 
PROMIS projects in the sites visited supported management information needs 
than did the non-PROMIS projects (e.g., across the six PROMIS sites, 70% of 
the management type applications were produced; across the 11 non-PROMIS 
sites, 38% were oroduced). 

Judgmental Assessment . Experience among PROMIS user jurisdictions 
has been that wide diversity still exists in terminology, data elements and 
procedures. Many of these differences are built into the legal and political 
structure of each jurisdiction and will not be changed by any information sys- 
tem. Substantial software changes have been made in modifying PROMIS in most 
jurisdictions in which it has been implemented. This requirement has been 
recognized by INSLAW in its latest MINI and MAXI versions of PROMIS which high- 
light a tailoring facility to simplify modifications of data definitions, file 
structures, etc. 

INSLAW produced a cross-city analysis of performance measures in 13 
jurisdictions using PROMIS. This effort took over two years to accomplish, 
although not on a full-time basis. On the scale carried out, it is a commend- 
able achievement, especially important for its focus on prosecutorial discre- 
tion and crime specific analysis. Even though each jurisdiction used PROMIS, 
the technical and data manipulation tasks were very difficult due to all the 
variations. There have been other cross-city analyses not relying on PROMIS; 
for example, the National Center for State Court's study of delay in 21 
general jurisdiction courts^ and a revealing study of the judicial process 
in three cities by Jacob and Eisenstein.^2 i n view of the potential for 
cross-city analysis, the surface has only been scratched by these publications. 
Questions remain as to the special need for PROMIS projects in such studies, 
as opposed to inclusion of other PMIS's, or the use of survey techniques by 
independent statistical agencies. 
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What is special about the INSLAW study is its focus on the exercise 
of prosecutorial discretion, the potential for crime specific analyses, and 
the working relationship built up between INSLAW and the district attorneys of 
the participating jurisdictions. This relationship probably could not have 
been developed- without the mutual experience of the LEAA-funded PROMIS transfer 
program. Over the years, INSLAW personnel . have demonstrated sensitivity to 
prosecutors concerns, competence in their technical fields, and understanding 
of the prosecution court processes and the pitfalls of inter-jurisdictional 
comparisons. If funding is continued, the cross-city program may result in 
the creation of a unique multi-jurisdictional time series data base for ■ 
analysis of prosecutorial discretion, court performance, policies, and impacts 
of various types of interventions. 

As a group, PROMIS projects make more use of management and statis- 
tical information than comparable non-PROMIS projects. Many of them have had 
cost and benefit studies performed by INSLAW in assessing the feasibility of 
their adoption of PROMIS projects. As a result of the above considerations, 
and common conceptual models, the PROMIS projects as a group should be easier 
to evaluate than non-PROMIS projects. 

b. ISSUE : To what extent are PROMIS projects more ‘or less c ost- 

effective than non-PROMIS projects? 

One of the assumptions LEAA has made in supporting technology trans- 
fer is the potential for saving development costs and improving systems quality 
and effectiveness. After more than six years of funding PROMIS transfer proj- 
ects, it seems appropriate to explore their cost-effectiveness relative to 
non-PROMIS projects. 



First, it is fair to recognize the extreme challenge inherent in 
attempting to develop a portable PMIS. There is great diversity among juris- 
dictions in their prosecution and court systems. Legal procedures, court 
organization, statutory constraints, management philosophy, resources, and 
staff capacities vary to such an extent that many question the basic concept 
of transfer at the software level. The ability to economically and conve- 
niently adapt or revise data definitions, data entry formats, data processing 
procedures and output displays is essential for software portability. 
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median development costs are in the range of $100,000 to $199,000. As shown 
in Table 9 above, this is comparable to the costs for non-PROMIS projects as 
compiled from the mail survey. 



One possible explanation for the lack of substantial development 
cost savings in transferring PROMIS is the extent of software modifications 
needed. Table 11 presents a tabulation of the opinions of PROMIS site person- 
nel on the degree of difficulty experienced in transferring PROMIS. More than 
half (13 of 24 agencies) considered transfer to be more difficult than expected, 
while none of the agencies considered it to be easier than expected. It is 
noteworthy that when one compares the different versions, the MINI- and MAXI- 
PR0MIS versions have a much higher ratio of "more difficult than expected" 
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Table 12. Cause of Difficulty in Implementing 
Operational and In-Transfer PROMIS 
Projects for Which Data Are Available ( N= 1 3 ) 



Primary Cause of Difficulty 



Software* 

1 

1 

i 

3 

1 



♦Software problems mentioned include bugs in programs and difficulty in 
compiling. 

• The survey data indicate that those who have operational PROMIS proj- 
ects are generally satisfied, although this situation is less so for F ROMIS II 
users. 

Judgmental Assessment. From the limited data collected on operational 
projects in Phase I, there Ts”Tit tie support for the theory that PROMIS projects 
are more cost-effective than non-PROMIS projects. The development costs do not 
appear to be less, possibly because of the necessity for substantial software 
modifications to accommodate inter jurisdictional differences, including among 
other factors, software/hardware incompatibilities. It may be true that future 
PROMIS transfers will be less costly than those to date because there are more 
hardware and software configuration versions from which to choose. As shown in 
Tables 6 and 7 above, there is also no evidence that PROMIS projects have been 
more effective than non-PROMIS projects, except possibly in providing manage- 
ment and statistical information. < 

c. ISSUE : What are the expectations for PROMIS projects? 

With the extensive promotional materials and support available, it 
is interesting to ask three questions about the expectations of PROMIS users. 

First, are their expectations unrealistic, leading to projects that are too j 

| N» 



Local 

Cooperation 



Redesign 
for Local 
Use 



Technical 

Reason 

Unknown 




PROMIS 

Version 

PROMIS 
PROMIS II 
MINI-PROMIS 
MAX I- PROMIS 
HYBRID 

TOTALS 
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ambitious with resultant disenchantment and low cost effectiveness? Second, 
and related to the. first question, how do the expectations of PROMIS users 
compare with non-PROMIS users? Third, what changes are occurring in the de- 
velopment and use of PROMIS projects, and what expectations are realistic 
for the future? 

Applicable Survey Data . Tables 6 and 7 compared expectations of 
PROMIS with non-PROMIS projects. Table 13 provides information on PROMIS proj- 
ects either desiring or planning changes. ' Of 23 operational or in-transfer 
PROMIS projects responding, 14 or 61 percent either desired or planned changes. 
The table shows that only 3 (27%) of 11 PROMIS I and II projects plan to stand 
pat. The largest swing is to MAXI-PROMIS; 7 (30%) of the 23 projects are 
switching to that system. 

Judgmental Assessment . With no clear evidence of cost and effective- 
ness advantages, why have so many prosecutors opted for PROMIS projects? It 
is apparent, from the Phase I site visits and telephone survey, that the type 
of user who is most enthusiastic about PROMIS projects is the chief prosecutor, 
that is, each jurisdiction's district attorney. INSLAW has enjoyed success in 
reaching top management in prosecutors' offices; this is by no means a trivial 
accomplishment. The PROMIS User's Group has been successful in providing for 
cross-city communication and exchange of insights on prosecution management 
policies, use of PROMIS management reports, and funding opportunities. The 
Phase I survey found that a relatively high proportion of PROMIS users experi- 
ence more difficulty than expected in transferring and implementing their 
systems, especially with software problems and data collection burdens. At 
the same time, from responses to the mail survey, it was apparent that prose- 
cution managers held high expectations for their PROMIS projects. 

One of the contributing factors to high expectations for PROMIS 
appears to be LEAA's continuing funding for PROMIS development. Indeed, since 
implementation of PROMIS, like implementation of many new software packages, 
often takes longer than planned, PROMIS transfer completions have occasionally 
•been outpaced by the release of enhanced versions of the system. For example, 
the highly publicized MINI-PROMIS package was released by INSLAW for field 
installation in December, 1978. In a telephone survey taken during September 
and October of 1979, which included sites listed by INSLAW as planning or having 
PROMIS projects, only one operational MINI-PROMIS as found. While this 9-month 
lag from release to operation is not in itself surprising, the survey revealed 
that very few of the sites were still planning to implement MINI-PROMIS. Many 
were switching to a still newer version, MAXI-PROMIS. 

MAXI-PROMIS, the latest version of PROMIS, was released by INSLAW 
in a preliminary stage in August 1979 for field installation. Of seven sites 
contacted who were in process of transferring MAXI-PROMIS, all reported major 
software modifications were needed. Systems personnel at these sites reported 
that they were doing extensive software development and modification to make 
MAXI-PROMIS work. As of October, 1979 no sites were found where MAXI-PROMIS 
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was operational. The system is still new and undergoing change. However, 
expectations remain high for the many attractive features of this new version, 
e.g., the tailoring and self-documentation features. 



Table 13. Changes Planned or Desired 
Operational and In-transfer 
PROMIS projects (N=23) 



Existing 

Version Totals 
PROMIS I 3 

PROMIS II 6 

MINI- 

PROMIS. 2 

MAXI- 

PROMIS 2 

HYBRID 8 

TOTALS 23 



Types of Changes 



To MINI- 


To MAXI- 


Add 


Add Appli- 


Add On-line 




PROMIS 


PROMIS 


Users 


cations 


Inquiry 


None 






0 





Source: Westat Mail/Telephone Survey, 1979 
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III. FEASIBILITY TESTS 



A. Introduction 



1. Evaluability of PMIS Projects ' , 

An LEAA report suggests three conditions which must be met For 
projects to be evaluable. 33 The conditions are: 

• Users of evaluation results must agree on definitions 
of activities, the conditions to be changed, and the 
kinds of expected outcomes. 

• The key project assumptions must be stated in terms 
which can be tested objectively. 

• Program or project managers must clearly define at 
least one use for evaluation information in making 
a decision or in initiating administrative action. 

Few projects surveyed in the first segment of Phase I had explicit 
statements of their goals and expected outcomes. However, there were enough 
models available through transfer projects and other well conceived PMIS 
projects to infer a set of implicit prosecution goals from interviews and the 
literature search. The evaluation framework, developed from observations in 
the first segment of Phase I and discussed in Section II C and in Volume I, 
presents what our generalizations of definitions of activities, conditions to 
be changed, and kinds of expected outcomes. 

Although the key project assumptions linking PMIS implementation to 
expected impacts were not explicitly stated in any project, the study enabled 
the research team to infer an implicit chain of assumptions. The evaluation 
•framework also specifies both quantitative measurements and judgmental assess- 
ments by independent observers as a means of objective evaluation. 

The final evaluability condition, identification of a clear use for 
the evaluation results, is the most significant factor. As stated earlier, 
large sums have been invested at the Federal, state and local levels in devel- 
oping and operating PMIS projects. Interest in PMIS development is expected 
to continue. However, particularly in light of recent Federal budget uncer- 
tainty, Federal, state and local government decisionmakers must allocate 
increasingly scarce funds to competing program areas. In the past, many PMIS 
decisions have been made based on intuitive, emotional, or political motiva- 
tions, rather than objective information. In the future, funding decision- 
makers and prospective PMIS project managers and users will almost certainly 
require a more complete understanding of the characteristics of successful 
PMIS development and operation, the situational variables that encourage or 
hinder successful PMIS implementation, and the expected results, costs and 
impacts. 




Analysis of the data collected during the first segment of Phase I 
indicated that all three conditions for evaluability appeared to be present 
and data required to measure performance/impact of PMIS projects appeared to 
be available. Thus, testing the feasibility of a Phase II evaluation design 
for PMIS projects was initiated. 




Due to the large number of projects surveyed in the first segment 
of Phase I, only a limited amount of time could be spent on each. Even where 
site visits were made, the project team had to rely on available reports and 
documentation or information gathered in interviews with a few key personnel 
on site. Thus, it was not possible to elicit detailed information considered 
crucial for a definitive evaluation of PMIS projects. One purpose of the 
feasibility tests, then, was to determine the actual availability of data 
needed for in-depth evaluations, and, where study team and/or PMIS project 
resources permitted, to collect that data for the purpose of testing evalu- 
ative, methods. 



Once data were collected in sufficient detail, tests were conducted 
to determine feasible methods of measuring and assessing PMIS impact on the 
prosecution and judicial processes, PMIS cost-effectiveness, and PMIS transfer 
potential. 



3 . Evaluation Approach 

Within the context of the general evaluation framework, the evalu- 
ator's specific approach was conditioned by two factors beyond his control: 
the age of the PMIS being evaluated, and the resources available to conduct 
the evaluation. Therefore, the project team tested the feasibility of both 
a priori and a posteriori evaluation designs (to be applied, respectively, to 
relatively new and relatively old PMIS installations), and the feasibility of 
evaluation approaches involving both intensive and non-intensive data collec- 
tion. 



For relatively new projects, the a priori design provides for 
thorough measurement of baseline conditions, and monitoring of changes in 
performance during the evaluation data collection period as measured by pre- 
defined indicators of success. For projects that have been operational two 
or more years, the a posteriori design focuses on the construction of a time- 
series from historical data files in a retrospective analysis of changes over 
time. At sites where system enhancement is occurring, these two approaches 
can be combined. For example, impact of operational support applications may 
be evaluated on an a posteriori basis while a newly introduced management/ 
statistics capability may be evaluated on an a priori basis. 

Each design perspective has advantages and disadvantages. On 
recently implemented projects, it is easier to reconstruct an accurate set 
of measurements of baseline conditions. Memories are' fresher for inter- 
views on situational variables; judgmental observations can be made, and 
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manual records on court performance are more readily available. The disad- 
vantage in a priori design is the limited time frame available for observing 
changes. For example, on newly implemented projects, only about six months 
of operation can be observed during the course of the feasibility study. Of 
course, this time frame will vary somewhat depending on the date the particular 
project became operational. > 

The a posteriori design has the advantage of providing about up to 
24 months of data in a time series format. Such data are amenable to con- 
structing more valid evaluative research designs to measure project impact. 

To the extent that gaps exist in the definition of baseline conditions, it is 
more difficult to reconstruct those from sources external to the PMIS itself 
because of failing memory, warehoused court records, etc. However, a properly 
constructed a posteriori design can yield the data necessary to measure 
project impacts. 

The a priori and _a posteriori approaches involve the use of both 
PMIS data and Samples of manual records. Resource constraints on the feasi- 
bility study precluded intensive data collection and analysis at all evalua- 
tion sites. Therefore, both intensive and non-intensive data collection 
approaches were used. 

The intensive approach, attempted in three sites, incorporated 
collection and analysis of PMIS-generated time-series data on indicators of 
success, as well as sampling and analysis of manual records to attempt to 
compute performance measures and to assess the quality of PMIS data. The 
non-intensive approach involved, instead, an assessment of the availability of 
PMIS time-series data to support impact evaluations and of the availability 
and comparability of manual records to support analysis of PMIS data quality. 
While use of the intensive approach at all sites would have been desirable, 
the non-intensive approach nonetheless generated valuable insights into the 
feasibility of PMIS evaluation. 

4. Test Sites 



Six PMIS projects were used as test sites for determining the 
feasibility of PMIS evaluations. Sites were chosen because of their charac- 
teristics and a preliminary indication of being an evaluable PMIS project.* 
Figure 6 depicts the division of PMIS projects by classes. This set of 
classes constitutes an initial division of PMIS projects for assessment of 
general characteristics of each group. The feasibility study was not intended 
to provide sufficient information for generalizing class characteristics; 
rather it was designed to test methods for developing such information. The 



♦Selection criteria for feasibility test sites and comments relevant to 
their selection are contained in Volume II, Final Report. 
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class numbers shown below are indicated by the numbers in the appropriate box 
in Figure 6. The sites selected within each class were: 



• Class 1 - Boston (Suffolk County), MA - Case Management 

System (CMS); 

• Class 2 - Oakland (Alameda County), CA - District Attorney 

Automated Legal Information System (DALITE); 

• Class 3 - Portland (Multonomah County), OR - MAXI-PROMIS 

• Class 4 - Golden, CO - Colorado Regional PROMIS (MAXI-PROMIS) 

• Class 5 - Norfolk, VA - Total Recall Adult Criminal Element 

Record (TRACER), a regional system; and 

• Class 6 - Oklahoma City, OK - Arrest Disposition Reporting 

System (ADRS) , a statewide system. 

The PMIS classes, the sites selected to represent each class, and 
the data collect ion/evaluation approaches are summarized in Exhibit 4. 



B. Availability and Quality of Data 
1 . Data Availability 

Survey teams spent three days collecting data at non-intensive sites 
and five days at the intensive sites. The primary survey teams consisted of 
two persons for each site visit. The survey teams visiting Norfolk and 
Oakland were each augmented by a Westat field representative. These field 
representatives were trained interviewers/ data abstractors employed to test 
the feasibility of abstracting case/defendant information from manual records. 

A data collection instrument was prepared for gathering information 
at the test sites. The collection instrument consisted of a separate section 
for each agency or office (prosecutor, courts, police and data processing 
facility) from which information was sought. Face-to-face interviews were 
conducted in each office or agency. The collection instrument was used as an 
interview guide and used to record data that were extracted from records or 
opinions expressed by the respondent. Exhibit 3 lists the general categories 
of information covered by the collection instrument and shows the availability 
of data within each category from the agencies (sources) at the six test' 
sites. As indicated in Exhibit- 5, most data sought in general information 
categories were either obtained during the surveys or it was determined that 
collection of such data is feasible. 



PMIS CLASS 



1 . Small minicomputer 



2. Large minicomputer 



3. Large scale computer 
limited data sharing 

4. Large scale computer, 
extensive data sharing, 
prosecutor/court 



SELECTED SITES 



Suffolk County, MA 
(CMS) 

Alameda County, CA 
(DALITE) 

Portland, OR 
(MAXI-PROMIS) 

Golden, CO 
(PROMT S/MAX I-PROMIS) 



DATA COLLECTION APPROACH 
Non- 

Intensivel/ Intensive!?/ 



EVALUATION APPROACH 
A Priori-?/ A Posteriori^/ 



5. Large scale computer, 
extensive data sharing, 
integrated CJIS 

6. Large scale computer, 
extensive data sharing, 
multi- jurisdiction 



Norfolk, VA 
( TRACER) 



State of Oklahoma 
(ADRS) 



- Collect data for time series analysis. 

- Determine availability of data for time series analysis. 
A P rior i - before PMIS implementation. 

- A posteriori - after PMIS implementation. 



Exhibit 4. Data Collection/Evaluation Approach 
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Exhibit 5. Availability of Data by General Category 



Actual Data Availability Statu» 



Category of Data Expected From 
Test Sites (Outline of Collection 
Instrument) • 


Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


Coldenf 


TRACER 


DALITE 


ADRS 


CMS 


PROMIS 


PROMIS 


PR 


CT 


PO 


DP 


Prosecutor 


PR 


SAC 


DP 


Prosecutor 


PR 


UP 


PR 


DP 


A. Personnel 

1. Number assigned 


r 


Y 


Y 


Y 


Y 


FS 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


. 2. Salaries 


Y 


Y 


Y 


Y 


Y 


FS 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


f B. Equipment 

1. Hakes, models, quantities 


Y 


Y 


Y 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


2. Costs 


Y 


Y 


Y 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


C. Input to PHIS 
1. Methods 


Y 


Y 


* 


Y 


Y 


NA 


Y 


NA 


. ' 1 


Y 


Y 


Y 


Y 


J* • 2. Volume 


Y 


Y 


Y 


Y 


Y 


NA 


F 




Y 


FO 


FO: 

i 


FO 


FO 


D. Output use and utility 


Y 


Y 


Y 


NA 


Y 


FS 


Y 


NA 


Y* 


F-0 


NA 


FO 


NA 


E. Data Quality 

1. User judgments 


1 

Y 


Y 


Y 


NA 


Y 


FS 


Y 


NA 


Y 


1 

F°, 


NA 


FO 


NA 


2. Comparison of PMIS data 
with manual records 

(** 


Y 


Y 


Y 


Y 


. Y 


NA 


F 


F 


F 


FO 


FO 


FO 


FO 


«C',. 

F. Prosecutor goals For PMIS 
1. Identification of Coals 


Y 


NA 


NA 


NA 


Y 


FS 


NA 


NA 


Y 


Y 


NA 


Y 


NA 


2. Prosecutor ratings of PMIS 
contributions to goal 
attainment 


Y 


NA 


NA 


NA 


Y 


FS 


NA 


NA 


Y 


FO 


NA 


FO 


NA 


_ G. Interventions 

f 


Y 


Y 


Y 


Y 


Y 


FS 


F 


F 


Y 


F 


N 


Y 


Y 


H. System operations 

1. Hethods and procedures 


Y 


Y 


Y 


Y 


Y 


NA 


Y 


v 


Y 


Y 


Y 


Y 


Y 


2. Costs 


Y 


Y 


Y 


Y 


Y 


NA 


Y 


Y 


Y 


FO 


FO 


FO 


FO 


1. Monthly caseload/caseflow 
statistics 


Y 


Y 


NA 


Y 


N 


NA 


, F 


F 


F 


FO 


FO 


FO 


FO 


J. PHIS development 
1 . Hethods 


NA 


NA 


NA 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


2. Costs 


NA 


NA 


NA 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


K, Transferability 

1. Hardware/software 
characteristics 

■ 


NA 


NA 


NA 


Y 


Y 


NA 


Y 


Y 


Y 


NA 


Y 


NA 


Y 


«t.V a 

2. Documentation status 


NA 


NA 


NA 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 



LEGEND: Y = Yes, data obtained 

N = No, data not available 

F = Feasible to collect data, not obtained at non-intensive sites 
FO = Feasible to collect data when system is tally operational 
FS = Feasible to collect data, needed on a statewide basis 
NA = Not applicable to agency, no attempt made to collect data 



•Agencies: PR = Prosecutor; CT = Court; PO = Police; DP = Data processing Facility; SAC = Statistical Analysis Center 









1; ) The availability of specific data and the sources from which such 

1 data can be collected varied among the surveyed sites. 

t 

1 Time Series Data 

l ’ 

j The availability of time series data is shown in Exhibit 6. Data to 

| | support time series analysis were sought from three sources: (1) PMIS histor- 

j ical files maintained by the ADP facility; (2) monthly statistical reports for 

j the preceding 24-36 months (either PMIS generated or manually prepared by .the 

prosecutor and courts); and (3) case files maintained by the prosecutor and 
I courts. 

1 

j I History tapes were obtained from Norfolk, Oakland and Golden. His- 

j tory tapes were not available from Boston because the project uses disk packs 

for secondary storage and no tape drive was connected to the system for use in 
outputting an historical file; the system in Portland was not yet operational; 
and, there was not sufficient time available to obtain approval from prosecu- 
tors statewide for the release of Oklahoma tapes. 

1 

Only the tape from Norfolk was used in testing evaluation methods, 
s because of time and money constraints. It was determined, however, that 

j appropriate data could be obtained from PMIS files at all test sites. 

| . 

. 
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Exhibit 6. Availability of Time Series Data 







Site Location/PMIS Acronym 




Norfolk 


Oakland 


Oklahoma 

City 


1 

Boston 


Portland 


Golden 


Data expected from test Bites 


TRACER 


DALITE 


ADRS 


CMS 


PROMIS 


PROMIS 


A. Computer history tapes 


Yes 


Yesr^ 


.. 2/ 
Nor- 


No 


No- 




B. PMIS generated statistical reports 
for the past 24-36 months 


Yes 

• 


Ncr^ 

. . 1 


Partial 


No 




No 


* 

C. Manually prepared statistical 

reports for the past 24-36 months 


Yes 


j 

No 


Partial 


No 


No' 


No 


D. Manually maintained case files 


! 

Yea 


Yes 


Yea 


Yes 


Yes 


Yes 


E. Prospective PMIS generated reports 
probably obtainable 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 



1/ 

y 

3/ 

y 

5/ 



History tapes from HP system could not be processed on IBM systems. 

History tapes exist and could probably be obtained, but would require authorization from prosecutors statewide. 

• * 

History tapes were made available from original PROMIS system; new PROMIS not yet operational. 

No statistical reports have been generated from DALITE; statistics generated by CORPUS are considered unreliable. 

PROMIS not yet operational; statistical reports are provided to the court in computer listings generated by the State 
Judicial Information System. 
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The feasibility of abstracting information from manually maintained 
case files, for the purpose of collecting time series data for periods prior 
to PMIS implementation, was also considered. Testing of abstraction proce- 
dures was conducted at Norfolk and Oakland. The time and cost involved in 
the data abstraction procedures are shown in Exhibit 7. The cost of extrac- 
ting data using these procedures averaged $1.57 per case. , 

Once needed data are abstracted, processing of that data is required 
to produce statistics for specific time periods (monthly, for example) to 
facilitate time series analysis. To accomplish this processing by computer, 
it would be necessary to enter the data into a computer record and then 
process that data to generate desired statistics. The cost estimated to 
accomplish this processing ^(s.ee Exhibit 8) is estimated at $.87 per case. 
Computer programming to accomplish this processing would be a one-time esti- 
mated cost of about $350 if "canned" statistical packages (such as SAS or 
SPSS) are to be used.* 

The cost to develop time series data for a jurisdiction such as 
Boston (where statistics are not available for the preceding 24-36 months), is 
estimated to be from $14,640 (24 months - 6,000 cases) to $21,960 (36 months - 
9,000 cases) based on approximately 3,000 cases per year at a cost of $2.44 
per case (for abstracting, entering, and processing the data). Considering 
the labor force for such an operation, the data abstraction on 6,000 cases 
would require approximately 215 person days; for 9,000 cases, 322.5 person 
days. 

Cost-Benefit Data 

. The purpose of collecting cost-benefit data was to determine the 
feasibility of performing cost-benefit analysis of systems representing the 
six PMIS classes. There was no intent to develop a model for cost-benefit 
analysis for this study as several models were available**, and others probably 
have been used for predicting cost or benefits in jurisdictions not included 
in this study. It seems appropriate that if an analysis is to be performed 
for a PMIS for which cost or benefits have been predicted, the model used for 
those predictions should again be used for that jurisdiction to permit compar- 
isons of common data (for before and after PMIS implementation), and for 
validating the model. 

Exhibit 9 shows the availability of cdst-benefit data at the test 

sites. 



* Programming costs will be substantially higher unless the analyst has 
structured the case abstract as a fixed-length record. 

**INSL AW cost-benefit model; MULTNOMAH COUNTY OREGON cost-benefit package; 
National Center for State Courts cost-benefit methodology for Evaluation 
of State Judicial Information Systems. 
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Exhibit 7. Abstracting Data from Manually Maintained Case Records 
Items regarding test sites Norfolk Oaklanc 



1. Number of records from which data were 
abstracted 

2. Total time (minutes) needed to abstract 
data (time needed to retrieve case file 
not included) 

3. Average time (minutes) per record (case) 
needed to abstract data 

4. Average wage rate ($ per minute) of 
person abstracting data 

5. Average cost per record (case) to 
abstract data 



Exhibit 8. Estimates Regarding the Processing of Abstracted Data 

Items involved in processing Estimates 

1. Average time (minutes) needed to enter on record (case) 

into PMIS 5.0 

2. Average wage rate ($ per minute) for data entry clerk • $ .060 

3. Average cost per record (case) for data entry $ .30 

4. Average monthly computer processing cost to produce all 

monthly statistics needed for time series analysis $97.00 

5. Average number of cases processed monthly as input to 

statistical reports 5,491 

6. Average cost per record (case) for computer processing 

of monthly statistical data $ .57 

7. Total estimated cost per record (case) to process 

abstracted data (data entry + computer processing) $ .87 
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Exhibit 9. Availability of Cost-Benefit Data 



Page i of 3 




Site Locat lon/PMIS Acronym 



Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


TRACER 


DAL t TE 


ADRS 


CMS 


PROMIS 



A. Cost Items 

1 , Total development 
costs for PMIS 



a. analysis, design, 
programming 



. documentation 



. equipment purchase 



d. other (testing, 
training, etc.) 



2. Local prosecutor's share 
of operating costs 



a. personnel costs 
(salaries ?*nd 
fringes) for local 
prosecutor's office 



b. equipment leasing 



c. equipment 
maintenance 



. communications 



e. other operations 

costs (depreciation 
of equipment, 
supplies, technical 
assistance) 



f., computer processing 
(charge for CPU time 



(1) on-line 

processing 
( inquiries/ 
entry) 



(2) batch 

processing 

(output 

reports) 



$222,240—' 



$ 54, IQ^ 



$ 18,532 



$ 63,360 



$ 12 , 34 ^ 




Legend: NA = Nat applicable 

No = Data not available 
Yes = Oata available 

F = Feasible to obtain data, not collected at non-intensive sites * 

FO = Feasible to obtain data after system is fully operational 

— ^Reqional systems: TRACER (Norfolk) serves multiple criminal justice agencies in multiple jurisdictions. 

.PROMIS (Golden) serves prosecutors in multiple jurisdictions. ADRS (OKLAHOMA) is a statewide system. 

—^Further breakdown of costs for individual items is available. 

^Operating costs were taken from* current budget documents ($27,539) plus the estimated annual cost for data entry ($2,8Hn) 
^Estimated for first year of operations. 

—PMIS serves prosecutors statewide; figures reTer to cost of PMIS operations as they apply to the Statistical Analysis 
, .Center (SAC) of the Oklahoma Crime Commission ns estimated in latest budqet request. 

— 'Norfolk ADP facility charges users according to CPU time. In determining charqeout rates, Norfolk indicates users share 
_,of personnel and equipment costs which equate to the costs Tor batch and on-line processing as shown on lines 2f (1) and (2) 
4/m in l computers surveyed do not maintain a record of CPU tlfcfc used for various transactions. 

— Central ADP facilities using large scale computers maintain records of CPU time usage by transaction; it is therefore 
^feasible to obtain these data. 

.^.Estimates not applicable to prosecutor, but can be made by other criminal justice agencies. 

— - F igures represent three year average. 
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Exhibit 9. Availability of Cost-Benefit Data (Continued) 



Cost-Benefit Items ' 



Site l ocat inn/PMIS Acronym 
llklnhoma 

nklarid City 



Page 2 of 3 



B. Benefit items - annual 
savings 

1. cost displacement 
(personnel, equip- 
ment or supplies no 
longer a recurring 
expense) 

Examples: 

• eliminate second 
(night) shift of 
clerks 



See 

examples 



None 

indicated 



None 

indicated 



None 

indicated 



None 

indicated 



See 

examples 



$ 49,000 
est imated 



None 

indicated 



eliminate f j ling 
cabinets, card- 
vendor, vistu- 
triever, etc. 



Est imateSg . 
possible-' 



$ 480 

estimated 



• reduce office 
supplies needed 
for manual records 
that are replaced 
by PMIS operations 

Cost avoidance 
(personnel, equipment 
costs not expended due 
to increased produc- 
tivity or efficiency) 

Ex amples: 

• PMIS generation of 
reports, warrants, 
subpoenas, etc., 
reduces clerical & 
typing time 

• reduce or eliminate 
need to period- 
ically increase ' 
staff to cope with 
increased caseload 



Estimates . 
possible-' 



See 

ex amples 



Estimates 

possible 



Estimates^ 

possible- 



bee 

ex amples 



None * 
indicated 



None 

indicated 



None 

indicated 



See 

ex amples 



None 

indicated 



None 

indicated 



Estimates 

possible 



See 

examples 



$ 12,000 

estimated 



See 

ex amples 



Est imates 
possible 



$ 25,36ol^ / None 



• reduce the need to 
obtain answers to 
case or defendant 
inquiries 

• reduce or eliminate 
need for periodic 
procurement of 
additional equip- 
ment to cope with 
increased caseload 



Estimates 

possible 



Estimates 

possible9/ 



Estimates 

possible 



Estimates 

possible 



$ 8,000 

estimated 



Estimates 

possible 



$ 1,70&15 / None 



Legend: NA = Not applicable 

No = Data not available 
Yes s Data available 

F r Feasible to obtain data, not collected at non-intensive sites 

FD r Feasible to obtain data after system is fully operational 

-^Regional systems: TRACER (Norfolk) serves multiple criminal justice agencies -in multiple jurisdictions; and 

2 .PROMIS (Golden) serves prosecutors in multiple jurisdictions. ADR5 (Oaklahoma) is a statewide system. 

-^Further breakdown oT costs For individual items is available. 

^Operating costs were taken from current budget documents ($27,539) plus the estimated annual cost for data entry ($2,880). 
^Estimated for first year of operations. 

—PMIS serves prosecutors statewide; figures refer to cost of PMI5 operations as they apply to the Statistical Analysis 
, .Center (SAC) of the Oklahoma Crime Commission as estimated in latent budget request. 

—Norfolk ADP facility charges users according to CPU time. In determining chargeout rates, Norfolk indicates users share 
7 ,of personnel and equipment costs which equate to the costs for batch and on-Une processing as shown on lines 2f (1) and (2) 
^Minicomputers surveyed do not maintain a record of CPU time used for various transactions. 

— Central ADP facilities using large scale computers maintain records of CPU time usage by transaction; it is therefore 
„ .feasible to obtain these data. 

^Estimates not applicable to prosecutor, but can be made by other criminal justice agencies. 

—Figures represent three year average. 
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Exhibit 9. Availability of 



i 



Cost-Benefit Data (Continued) 

Page 3 of 3 





Site Locat lon/PMIS Acronym 




Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


Golden 


Cost-BeneFit Items 


TRACER 


DALITE 


ADRS 


CMS 


PROMTS 


PROMTS 


3. Value added (improved 
services, performance, 
and decisionmaking 
capabilities) 


See 

exampJ.es . 


See 

examples 


See 

examples 


See 

examples 


See 

examples 


See 

examples 


Ex amples: 

• more inquiries can 
be answered than 
with manual system - 
better service to 
public and users 


Judgment 

possible 


Judgment 

possible 


None 

indicated 


Judgment 

possible 


$ 5,000 
judgment 


Judgment 

possible 


. new statistical 
reports - aid 
decisions 


Judgment 

possible 


Judgment 

possible 


Judgment 

possible 


(Judgment 

possible 


$ 22,000 
judgment 


Judgment 

possible 


• case ratmg/case- 
weightmg capability 
enhanced 


None 

indicated 


Judgment 

possible 


None 

indicated 


None • 

indicated 


$146,250 

judgment 


None 

indicated 


• more efficient use 
of resources 
through improved 
scheduling and 
notifications to 
appear in court 
(continuances 
reduced) 


Judgment 

possible 


Judgment 

possible 


None 

indicated 


Judgment 

possible 


Judgment 

possible 


Judgment 

possible 



Legend: 



1/T 



NA = Not applicable 
No = Data not available 

Yes = Data available 

F = Feasible to obtain data, not collected at non-intensive sites 

FO 3 Feasible to obtain data after system is fully operational 



— Regional systems: TRACER (Norfolk) serves multiple criminal justice agencies in multiple jurisdictions-; and 

.PROMTS (Golden) serves prosecutors in multiple jurisdictions. ADRS (Oklahoma) is a statewide system. 

^Further breakdown of costs for individual items is available. 

j-^Operat mg costs were taken from current budget documents ($27,539) plus the estimated annual cost for data entry ($2,880). 
•^Estimated for first year of operations. 

— 'PMIS serves prosecutors statewide; figures refer to cost of PMIS operations as they apply to the Statistical Analysis 

v! 



Center (SAC) of the Oklahoma Crime Commission as estimated in latest budget request. 



— Norfolk ADP facility charqes users according to CPU time. In determining* chargeout rates, Norfolk indicates users share 
.of personnel and equipment costs which equate to the costs for batch and on-line processing as shown on lines 2f (1) and (2 
•^Minicomputers surveyed do not maintain a record of CPU time used Tor various transactions. 

—Central ADP facilities usinq large scale computers maintain records of CPU time usage by transaction; it is therefore 
.feasible to obtain lihese data., 

.-^•yEst imates not applicable to prosecutor, but can be made by other criminal justice aqencies. 

— / fiqures represent three year average. 
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Development costs were taken from grant documents and records that 
account for expenditures for various development components such, as analysis, 
design, and programming. 

•Operating costs were taken from current budgets for those systems 
that were fully operational at the time of. the survey (Norfolk, Oakland, 
Oklahoma City, and Boston). Estimates of operating costs were taken from 
a cost-benefit package prepared by Portland. Operating costs for the old 
PROMIS system in Golden were obtained, but have not been included in Exhibit 
9 which refers to new PROMIS. 

Benefits could fee "identified for the fully operational PMIS proj-v 
ects and predictions of the .value of benefits have been made for the two 
PROMIS projects (Portland and Golden). Portland used the Multnomah County 
cost-benefit model to predict benefits while the INSLAW developed cost- 
benefit model was used for Golden. The model used by Portland defines 
benefits in general terms allowing for the recording of specific benefits in 
an open-ended manner. This format is conducive to illustrating the avail- 
ability of cost-benefit data at the test sites and has been used as the 
outline of benefits listed in Exhibit 9. 

» 

Exhibit 9 shows the cost savings predicted by Portland. The 
comment "estimates possible" indicates that a fairly accurate cost savings 
can be attached to the defined benefit; "judgment possible" indicates that 
cost savings attached to the defined benefit would represent a "best guess" 
by experienced personnel; and "none indicated" means that there was no 
indication that the defined benefit is applicable to the PMIS surveyed. In 
general, value estimates are more generally available for benefits in the 
"cost displacement" and "cost avoidance" categories than in the "value added" 
category; therefore, cost/benefit evaluation becomes more judgmental as the 
mix of benefits includes more of the latter group. At one end of the spectrum, 
Portland, anticipating several types of cost displacement and cost avoidance 
benefits, has completed a cost/benefit analysis largely in financial terms. 

For Oakland and Boston, and particularly for the Oklahoma statistical system, 
the benefits are almost completely of the value-added type. The fact that 
this makes cost/ benefit analysis of these systems more subject to judgment 
does not imply that the systems are less desirable - than the others. 

Data Concerning Transfer Potential 

A number of environmental and system factors may influence the 
transfer potential of a PMIS. Exhibit 10 shows the data collected at the 
test sites regarding PMIS environment and characteristics. 

Detailed cost data on the analysis, design, and programming compo- 
nents of PMIS development were not available at all sites: Oakland and Boston 

did not identify these costs; and Oklahoma City, Portland, and Golden could 
calculate these costs fairly accurately. Norfolk, on the other hand, had 
complete cost details on all developmental components. Since any jurisdiction 
considering acquisition of a PMIS by transfer will incur costs relating to all 
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Exhibit 10 . PMIS Environment and Characteristics 

Page 1 of 2 



Site Locat ion/PMIS Acronym 



Environment and Characteristics 
Descriptors 


Norfolk 


Oakland 1 

- 


Oklahoma 

City 


Boston 


Portland 


/ 

Golden 


TRACER 


DALITE 


ADRS 


CMS 


PR0MIS 


PR0MIS 


A. Environment 

1. Area served by PHIS 


Region 


County 


State 


County 


County 


Region 


2. Population served 


1,250,000 


1,100,000 


2,766,000 


723,000 


1,000,000 


1,500,000 
















3. Number oF local 
prosecutors 


16 


120 


316 


106 


60 


25 


A. Local prosecutor's 
annual caseload 


2,90ol / 


8,8Q0i / 


50,000 


3 .OOol/ 


12,200 


2,OOq1 / 


5. Agency controlling 
computer operations 


City ADP 


Prosecutor 


County ADP 


Prosecutor 


County ADP 


County ADP 


6. Extent oF PMIS usage 


multiple 
agencies and 
jurisdictions 


prosecutor 

only 


multiple 
agencies and 
jurisdictions 


prosecutor 

only 


prosecutor 

only 


prosecutors 
oF multiple 
junsdict ions 


B. Characteristics 

1. Date MIS became 
operat lonal 


7/76 


10/74 


1/77 


11/77 


not 2 / 

operational— 


1/80 


2. Method oF PMIS 
development 


in-house/ 

contractor 


in-house/ 

contractor 


in-house 


in-house/ 

contractor 


in-house/ 

transFer 


in-house/ 

transFer 


3. Cost oF development 


$158,880 


$ 42,185 


- 


$ 73,985 


$136,900 


$172,722 


4. Cost oF equipment 


$ 63,360 


$222,815 


- 


$ 63,800 


$ 3,100 


$233,694 


5. Total developmental cost 


$222,240 


$265,000 


$248,000 


$137,785 


$140,000 


$406,416 


6. Annual operating cost 
For entire PMIS 


$270,768 


$155,821 


$250,000 


$ 33,600 


$ 65,678 


- 


7. Local prosecutor's 
annual operating cost 


$ 30, 41^ 


$155,821 


NA 


$ 33,600 


$ 65, 678^ 


- 


8. Hardware 

a. size oF computer 


Large 


Mini 


Large 


Mini 


Large 


Large 


b. central processing 
unit 


IBM 370/145 


HP3000/III 


IBM 370/158 


Nova 2/10 


Amdahl 270 


Honeywell 662F 


c. terminals used by 
prosecutor 
(make-type) 


I MB - CRT's 


HP - CRT'S 


RACAL - MILG0 
CRT's 


Data 

Terminal, 
Inc. - CRT's 


IBM - CRT's 


Hazelt me 
CRT's 



—'Felony cases. 



2 / 

— For the purpose of this study, a PMIS was considered operational if data were being entered and iF outputs (visual displays 
in response to inquiries and/or printed reports) were being produced at the time oF the survey. 

-^Operating costs were taken From current budget documents ($27,539) plus the estimated annual cost For data entry ($2,08G). 
A/ 

— Operating costs are estimates For the First year oF operation, taken From the cost-beneFit package. 
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Exhibit 10 . PMIS Environment and Characteristics (.Continued) 

: Page 2. of 2 



Site Locat ion/PMIS Acronym 



Environment and Characteristics 
Descriptors 


Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


Golden 


TRACER 


DALITE 


ADRS 


CMS 


PR0MIS 


PR0MIS 


9. Software 

a. operating system 


DDS/VS 


MPE - III 


DOS 


MIC0S - II 


0S/VS1 


GEOS 


b. data base 

management system 


CICS - VS 


IMAGE 3000 


IMS 


None 


ADABAS 


I-D-S 


c. teleprocessing 
monitor 


CICS 


MTS 3000 


CICS 


None 


COMPLETE 


TDS 


d. programming 
language 


ANSI COBOL 


ANSI 

COBOL 74 


ANSI COBOL 


BASIC 


ANSI 

COBOL 68 


ANSI 

COBOL 74 


10. Input method 


on-1 ine 


on-line 


on-line by 5 
counties, 
forms mailed 
by others, 
then on-line 
by SAC 


on-line 


on-line 


on-line 


11. Processing capabilities 
a. on-line inquiry/ 
response 


Yes 


Yes 


Yes 


Yes 


Yes 


Planned 


b. on-line printing 
(e.g. , forms, 
notices) 


Yes 


Yes 


Yes 


Yes 


Yes 


Planned 


c. on-line report 
generation 


Yes 


Yes 


No ! 


Yes 


Yes 


Planned 


d. oFF-lme report 
generat ion 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


12. Types of support and 
reports 

a. on-line booking 


Yes 


No 


No 


No 


No 


No 


b. on-line arrest 
reports 


Yes 


No 


No 


No 


No 


No 


c. on-line warrants 


Yes 


No 


No 


No 


Yes 


Yes 


d. schedules 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


e. workloads 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


f. statistics 


Yes 


No 


Yes 


Planned 


Yes 


Planned 


13. Documentation status 
a. general system 
description 


Complete 


CompleLe 


Complete 


Complete 


Complete 


Complete 


b. system design 


Complete 


None 


Complete 


None 


Complete 


Complete 


c. program source 
listings 


Complete 


Complete 


Complete 


Complete 


Complete 


Complete 


d. logic flow charts 


Complete 


None 


Complete 


None 


Complete 


Complete 


e. operational manual 


Complete 


None 


Complete 


Comp lete 


Complete 


Complete 


f. users manual 


Complete 


None 


Complete 


Complete 


Complete 


Complete 



— Felony coses. 

2 / 

— For the purpose of Ihis study, a PHIS was considered npernl. lonnl |T outputs were heinil produced ol 

■^Operating costs were taken from current buriqet documents ($27,539) plus the estimated annual cost 
4/ 

— Qperatinq costs are estimates Tor the First year oF operation, taken From the cost-benefit packnqe 



the time oF the survey, 
ror data entry ( $2 , BAD) . 
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aspects of system development, it would be helpful to be able to compare costs 
of each development component between transfer systems and those fully devel- 
oped at a jurisdiction. 
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The data collection effort in the field investigations included 
three types of tasks. First, the project itself was described, e.g., data 
on staffing, costs and computer facilities. Second, the MIS was described, 
including its operational capacity and interventions over time, its inputs, 
and its outputs. Third, it was desirable to collect sufficient data on the 
operation of the criminal justice system, e.g. , monthly caseload and dispo- 
sitions, to permit the construction of a time series data base. 



As expected, much of the data collection was straight forward. The 
three types of data that were most difficult to obtain were comparable project 
costs, external checks on the accuracy of computer data, and time series data 
on criminal justice system performance. Thus, the assessment of feasibility 
was most concerned with these three data collection issues. 



Given the collection of adequate time series data, another major 
issue was the availability of appropriate techniques for analyzing the impact 
of the prosecution MIS. These techniques had to be able to measure changes in 
criminal justice system performance in the time frame of the PMIS interven- 
tions. In addition to detecting changes in performance patterns, techniques 
were needed to assist in attributing any changes to the PMIS interventions. 
Thus, data were required to describe significant variations in caseload mix, 
personnel, policies, organization, facilities, and other factors that could 
affect performance in addition to the PMIS itself. 

Commonality of Data Across Test Sites 

Common data were available from all test sites regarding: 

• Personnel - their salaries, fringe benefits and their work- 

load relevant to PMIS operations 

• Equipment - makes, models, quantities, cost of purchase or 

lease and maintenance costs 

• Operation of PMIS - methods of input, processing, and output; 

use, utility,, and quality of outputs 
(from fully operational sites); hardware 
and software characteristics; and status 
of PMIS documentation 



• Development of PMIS - methods of development. 
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Data lacking commonality among sites includes: 

• Caseload - caseload data differs within jurisdictions as well 

as among jurisdictions. In Norfolk, for example, 
caseload statistics maintained by the prosecutor 
represent "adult . felony indictments"; in the 
Circuit Court, the caseload figures represent 
counts (there may be several counts in an indict- 
ment) of felonies and misdemeanors; and the District 
Court statistics represent the number of hearings 
held. Caseloads do not necessarily represent work- 
loady^s far as PMIS operations are concerned. In 
- all .test sites except Norfolk, the prosecutor (or 
SAC ItT Oklahoma City) is responsible for entering 
all data pertaining to a case or defendant. In 
Norfolk, the police enter arrest data and the courts 
enter disposition data. It is important, then, to 
define the manner in which caseload counts are made; 
if cost per case is a factor to be considered, a 
common denominator for counting "cases" should be 
used. * 

• Cost of PMIS Development - developmental cost data vary not 

only in degrees of detail, but also in components 
included (e.g., contractor cost only or in-house 
staff as well) and treatment of "soft" costs (e.g., 
management efforts by chief prosecutor). Moreover, 
when comparing development costs per case, report, 
inquiry, population, and prosecutors served across 
jurisdictions, it is important to take note of the 
area served by the overall PMIS. In systems serving 
regional (Norfolk) and statewide (Oklahoma City) 
areas, it is difficult to allocate the cost of 
development for one specific prosecutor's office. 
Therefore, in some sites, developmental costs were 
stated in gross terms and not equated to the support 
provided one prosecutor's office. In Golden, on the 
other hand, the grant application shows expected 
development costs for each district. 

• Operational costs - total costs of operation can be determined 

for each PMIS. Cost backup data leading to those 
totals vary among PMIS projects. Centralized ADP 
facilities, such as Norfolk, Portland and Golden, 
charge prosecutors for ADP services. In Norfolk, 
the prosecutor is charged for his share of PMIS 
operations based on his usage of CPU time. The rate 
per CPU minute is set to cover the costs of ADP 
personnel, ADP equipment, and ADP equipment depre- 
ciation. In Portland and Golden, charges are made 
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according to CPU time used plus the cost of personnel 
time. In Oklahoma City, SAC budgets for PMIS opera- 
tions and individual prosecutors are not charged for 
the service. Cost of operations for prosecutor owned 
systems (Oakland and Boston) involve primarily direct 
costs for personnel, equipment rental, and supplies- 1 
devoted to the PMIS. In the latter cases, realistic 
costs per transaction (input, inquiry, and report) 
could not be determined within the available time 
and resource constraints, thus precluding the com- 
parison of such costs across all test sites. 

3. Comparison of Expected Versus Actual Availability of Data 

It was expected that caseload and caseflow statistics would be more 
readily available; if not being produced by the PMIS, it was expected that 
manually kept records would be on hand. Monthly statistical reports are 
currently PMIS generated only by Norfolk and Oklahoma City; such reports are 
planned for Boston, Portland, and Golden. Only one manually prepared statis- 
tical report (from the Norfolk Circuit Court) was available. 

More cost-benefit data were expected to be available. Cost savings 
have been predicted for only two sites using cost-benefit models; none of the 
other four sites have documented either predicted or actual savings. 

It was expected that more data would be available regarding new- 
PROMIS operations in Golden. INSLAW newsletters and contacts with site 
personnel indicated that the system was "operational." As used by INSLAW, 
"operational" means that the software has been installed on the hardware 
system, and data are being entered into the systems. At the time of the site 
visit, between 500-700 cases had been entered into the Golden new PROMIS 
system and inquiries could be made on those cases; however, no scheduled out- 
put reports were being produced. Also, the proportion of cases entered to 
date was so small that the system was not being used in day-to-day operation. 
Except as indicated above, the availability of other data was as expected. 

C. Analysis of Test Data 

1 • Cost-Benefit Analysis 

Quantifying PMIS costs is a relatively easy task compared to quan- 
tifying PMIS benefits. Only Portland identified a reduction in prosecutor's 
staffing and equipment that is expected because of PMIS implementation; 
these benefits, if reductions actually take place, can be easily quantified. 
None of the other prosecutor's offices could identify such clearly defined 
benefits. In Norfolk, however, the police and the courts are active, on-line 
users of the system and those agencies can identify and quantify extensive 
benefits directly related to the PMIS. 
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One important PMIS benefit identified at all sites is the improved 
capability to track cases. The fact that cases will no longer "fall throuqh 
the cracks (an expression used repeatedly), or "fewer cases are dismissed 
because of age," are improvements indicated by the prosecutor. The PMIS query 
and response capabilities reduce the burden of manually retrieving case and 
defendant information, a PMIS benefit of concern to line prosecutors and the 
support staff. More efficient use of personnel time, a result of improved 
scheduling using PMIS capabilities, is another benefit often cited by prosecu- 
tion personnel. . 

Benefits derived from new reports, which would not be economically 
•feasible without the PMIS, are-difficult to define and may be impossible to 
quantify in many jurisdictions™ . Portland has used the method of estimating 
the cost of manually producing new reports and subtracting from that the 
estimated cost of PMIS report generation to determine a cost savinqs. This 
method may be valid for certain types of reports (for example, statistics to 
satisfy state reporting requirements) , but other factors should be considered 
in determining the value of new reports. A report, although similar or iden- 
ticaliri content, may be used differently among jurisdictions or even within 
a jurisdiction. In Norfolk, for example, the Circuit Court used TRACER case 
aging and caseload reports to aid in identifying problems regarding the 
time involved in case processing; the result was that less court time is now 
being spent on felony 1 and 2 cases. Similar benefits were not identified 
nt-i • P rosecutor ' s office, although his office receives similar outputs. 

llization of the TRACER outputs differ between the prosecutor and the 
court; and the value placed on those outputs would therefore differ between 
he .two agencies. In the prospective INSLAW cost/benefit analysis prepared 
or Golden, this difficulty is addressed by use of "willingness to pay" as 
the evaluation measure; however, even that measure is difficult to assess in 
advance of PMIS implementation. 

Benefits can be identified and quantified where PMIS outputs replace 
manually generated products such as warrants, subpoenas, witness notices, and 
scheduled reports. Clerical/secretarial personnel can give fairly accurate 
estimates of the time required to manually produce such items, so costs of 

?K nU pm. P f° dUCtl0n Car I be determined fa y knowing the wage rate of the producers. 
The CPU time required to by the PMIS to produce these items can be determined 
and costs derived from the rate charged for CPU usage. A valid estimate of 
cost savings can be made for those functions actually replaced by PMIS- 
operations. - ^ J 

‘ ,, ®® ne fit s change over time. In the Norfolk situation, discussed 

above, the benefit of decreasing case processing time occurred approximately 
20 months after PMIS implementation; the benefit resulted from the initiation 
of new reports as part of a phased plan for implementing PMIS support. 

Other factors will effect the time when PMIS benefits occur: 

• In Portland, for example, dual' operations (PMIS and manual) 
are planned for a 6-9 month period, and dual operations 
took place in Boaton for an extended time. 
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• A "learning curve" will be involved for prosecutors who 
receive management-type reports never before provided to 
them, and benefits will accrue only after experience has 
been gained in the utilization of these reports. 

• Benefits expected or benefits actually accrued may be > 

reduced or negated by management decisions made after the 
PMIS has been operational for some time. In Norfolk, for 
example, two police booking stations were consolidated 

into one because of the on-line booking capability of 
TRACER, resulting in a substantial savings in manpower 
and equipment. Now, because of transportation problems 
and fuel consumption, the police are re-opening the second 
booking station, thus reducing, to a large degree, a PMIS 
benefit. Other examples include jurisdictions that continue, 
or revert to, manual recordkeeping because of inaccuracies 
in PMIS outputs, although the PMIS continues to generate the 
outputs; the PMIS generated report may have originally been 
considered a benefit, but has emerged as an added cost in 
such situations. 

• 

In those jurisdictions where technical resources are limited (Oak- 
land and Boston, for example), benefits are slow in developing because PMIS 
personnel must devote full time to daily operations. In both jurisdictions, 
the PMIS is primarily a case/defendant tracking system with printed outputs to 
support scheduling, case assignments, and retrieving of information. Both 
jurisdictions plan to have statistical reports but neither jurisdiction will 
reap benefits from such outputs for some time. The quantification of benefits 
at these sites would depend, primarily, on the value judgments of the prosecu- 
tors because little in way of personnel and equipment cost savings can be 
identified. It was interesting to note, however, that the Oakland PMIS had 
an impact on legislation* and that the Boston PMIS has proven helpful to the 
police**, which are examples of other benefits that would require judgments 
to quantify — or possibly cannot be quantified at all. Even though very few 



*DALITE was used by the Alameda County prosecutor’s office to aid in the 
research of cases involving a "Ballard Motion" which requires rape victims 
to submit to psychiatric treatment prior to trial. Cases that involved a 
motion (the Ballard Motion, per se, could not be identified by the system) 
were selected and listed by the PMIS. The prosecutors performing the 
‘research then retrieved file jackets for only those cases on the list, pre- 
cluding the need to look at all file jackets for motions. The research 
revealed extensive delays in trial proceedings because of the Ballard Motion. 
The resulting court decision on the matter led to repeal of the Ballard 
Motion legislation. 

**0ne Boston police investigator uses the PMIS generated schedule of cases to 
identify persons scheduled to appear in court who are "friends of fugitives." 
On the day of trial, the investigators visit the court to see if the fugi- 
tive shows up as a spectator at his friend's trial. A number of fugitive 
arrests have been made using this procedure. 
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quantifible benefits can be identified for these systems, the prosecutors are 
enthusiastic about the improved capabilities provided by the PMIS and, at 
this point, certainly consider the system to be worth the cost. 

•The main points emerging from this analysis of cost-benefit data 

are: 

/ 

• A PMIS may provide benefits to more than just the prosecutor, , 

even in those jurisdictions where the prosecutor has sole 

access to the system, and particularly where the system is- 
shared by other criminal justice agencies. An analysis of PMIS 
costs and benefits should, therefore, include the examination of 
capabilities and outputs provided to all agencies concerned. 

• A number of benefits can be identified that may be difficult 
or impossible to quantify. Some benefits can be quantified 
merely by comparing the cost of manual versus PMIS functions; 
while other benefits require a judgment of their value by the 
users, for which there is no standard measurement. 

• The results of a cost-benefit analysis may be iftvalidated by 
subsequent events that violate the assumptions of the analysis 
and affect the magnitude of a given cost or benefit element. 

2 . Performance Measures 

Of the six sites selected for field investigations, two were con- 
sidered as a priori evaluation sites because their PMIS's or certain key 
applications were not operational over a sufficient period of time.* Port- 
land's MAXI or new PROMIS system was not yet operational** at the time of the 
site visits and Boston's statistical and management reporting capability was 
not yet operational. In these two sites, the feasibility study focused on the 
issue of constructing a baseline on criminal justice performance and following 
changes prospectively. 

In the four sites selected for a posteriori analysis, the intent was 
to construct a data base spanning at least two years, consisting of monthly 
summaries of case dispositions, delay, and other pertinent performance mea- 
sures identified in the evaluation framework. It was also desired to include 
data on control 'variables, such as caseload and type of case. The field 
investigation included an examination of alternative methods of data collec- 
tion, e.g., derivation of data from processing of computer files, tabulations 
drawn from copies of previously produced monthly statistical reports, and 
tabulations drawn from a manually retrieved sampling of court case files. 

* Operational two or more years. 

**For the purpose of this study, a PMIS was considered operational if data 
were being entered and if outputs (e.g. , visual displays in response to 
inquiries and/or printed reports) were being produced. 
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A data base consisting of 36 monthly summaries was constructed for 
the city of Norfolk, starting with January 1977, or six months after the PMIS 
became operational. The following performance measures were computed for 
felony 1, felony 2, and other felony cases: 

• number and rates of cases terminated, by plea; < 

■ number and rates of cases terminated, by court dismissal; 

• number and rates of cases terminated, by nolle by prosecutor; 

• number and rates of cases terminated, by trial; 

• mean days from arrest to indictment; 

• mean days from indictment to trial; 

• mean days from trial to sentencing; 

• number of fugitives added; . 

• number of fugitives apprehended; and 

• cases set for trial. 

In addition, certain workload variables were available, including 
the number of cases commenced: type, number of hearings, and jury-trial days. 

A measure of court backlog was* available through a special manual system 
maintained by the court administrator, but was not kept in the computer 
system. 

In order to discern patterns of change, each performance measure 
was plotted against time in producing a two dimensional graph. To facilitate 
visual comparisons of the relative magnitude of change across the various 
plots, the values on the vertical scale (i.e., the impact measures) were 
standardized to vary from -2 to +3, with some outliers. 

Several performance measures exhibited a change in pattern following 
this PMIS implementation. The number of felony 1 and 2 cases terminated by 
guilty plea had been declining between January, 1977 (time period 1 in the 
data) and March, 1978 (time period 15); then appeared to start increasing in a 
parabolic fashion as depicted in Figure 7. A possibly related pattern of 
change was observed for- mean days from indictment to trial; this pattern 
changed from a rising one to a leveling off around the 15th month, as shown in 
Figure 8. The third variable for which a change in pattern occurred also 
would appear to be related. A rising pattern in the number of felonies dis- 
missed by the court, other than felony 1 and felony 2, was observed starting 
about the 14th month (see Figure 9). These three patterns could be consistent 
with a policy of reducing backlong and delay of serious cases through more 
intensive plea bargaining and court dismissals. • This hypothesis was examined 
during the impact analysis described below. 





88 




T 






► 



I 



ft 



t 



€ 



© 



' ' t 

It was desirable to attempt to adjust performance measures to 
indicate relative efficiencies, that is to divide by caseload volume. We 
would like to have divided each month's figures by the backlog (defined as 
cases pending for prior month, plus cases commenced, less cases terminated). 
However, because the backlog of pending cases was not available in the 
computer system, a surrogate measure, cases commenced, was used to adjust 
for workload. 

The analyses indicated no substantial change in patterns of per- 
formance over the 36 month period for many of the measures, whether or not 
account was taken of the court's workload, as indicated by cases commenced. 
For example, Figure 10 depicts the pattern observed for the proportion of 
felony cases terminated by trial. 

Additional data for measuring performance are maintained on 
Norfolk's TRACER system, but these data were available for only a portion of 
the 36 month time series period. These data include: 

• Trial outcomes — guilty, not guilty 

• Convictions — on original charge, or reduced charge 

• Cases terminated — filed this term, filed five months or less 
3. Impact Analysis 



The measurement of the impact of the PMIS on the p^osecution/court 
process involves two aspects of analysis — descriptive and explanatory. 

First, changes in patterns of prosecution/court performance need to be 
detected. Second, the analysis must attempt to determine whether the change 
can be attributed to the PMIS. The latter explanatory analysis attempts to 
rule out effects of external events such as policy or personnel changes. 

Each variable was plotted over the 36 month time frame. The pattern 
of variation in several of the performance measures exhibited a change around 
March, 1978, the 15th month. Thus, a change in performance was observed. 

This raised the question — did the change arise due to the use of TRACER? As 
indicated in the discussion on the performance measures, the observed changes 
could have been due to the change in chief judge and the institution of a 
master calendaring procedure for assigning judges to cases. According to the 
clerk of the court, it also could have been partially due to the improved 
utilization of TRACER. This finding would be consistent with a hypothesis 
that after a 15-month "break-in" period TRACER assisted the prosecutor and 
court in monitoring cases more efficiently; thereby increasing pleas and 
reducing time delays. Insufficient information was available to decide this 
aspect of the analysis. 

In order to identify and measure any impacts of the PMIS, it is use- 
ful to hypothesize a set of expected impacts.' Such hypotheses were formulated 
for Norfolk's TRACER system as part of the Evaluation Framework. Of course, 
the hypotheses actually tested were limited by the availability of data. 
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One hypothesis was discussed above, i.e., that improved case 
tracking added by TRACER enabled the increase in guilty pleas for felony 1 
and 2 cases and helped the court reduce time delays. In order to further 
examine possible impact, a productivity index was hypothesized. This index 
was structured' to combine measures of efficiency and effectiveness. It was 
hypothesized that productivity would rise as the impact of the PMI5 (TRACER, > 
in this case) was felt. The productivity index was assumed to vary directly 
with cases terminated by plea, cases terminated by trial, cases set for trial, 
and fugitives apprehended. It was assumed to vary inversely with mean days 
from arrest to trial, cases nolled and dismissed, and fugitives added. These 
assumptions were tested for each class of felonies by means of factor analysis. 
Following validation of these assumptions, the factor loadings were used to 
construct the productivity index. The index, in turn, was used to test the 
hypothesis that the PMIS improved productivity. 

Seven variables were selected for inclusion in the factor analysis, 
consistent with the hypotheses to be tested, the availability of data, and 
examinations of preliminary data plots. The seven variables were total cases 
terminated by plea, by dismissal and by trial, fugitives added, fugitives 
apprehended, cases set for trial, and mean time from arrest to trial. The 
factor analysis had the effect of reducing the number of variables from seven 
to three. 



The three factors are summarized below, indicating those variables 
which are highly correlated (.7 or higher factor loading): 

(1) Factor 1 (positive productivity) 

Cases terminated by plea - factor loading .887; 

Cases terminated by trial - factor loading .768; 

Cases set for trial - factor loading .744; 

(2) Factor 2 (negative productivity) 

Cases terminated by dismissal/nolle - factor loading .696; 

Fugitives added - factor loading .814; 

Fugitives apprehended - factor loading .872, and 

(3) Factor 3 (delay) 

Mean days from arrest to trial, factor loading .949. 

Plots of each factor against time were then performed to attempt to 
detect any changes in patterns, especially in testing the hypothesized impact 
relationships.- 
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The fluctuations in Factor 1 identified with positive productivity 
showed very little change in trend over time. A similar lack of any identi- 
fiable impact of the PMIS (see Figure 11) resulted when the variables were 
converted to rates, i.e. , divided by the corresponding figures for cases com- 
menced. The fluctuations in Factor 2 associated with negative productivity 
variables also appeared to be random either in unadjusted form (see Figure 12) 
or when adjusted for workload. The delay factor also showed no substantial 
change in pattern. Thus, there was no support for any hypothesized impact 
relationships on the total caseload. However, it remained to be tested 
whether impacts were masked by lumping all felonies together. 

A finding that there was little impact of TRACER on the court and 
prosecution system would have been consistent with certain observations during 
the site visits. TRACER was being used by the prosecutor for case and defend- 
ant status monitoring via terminal inquiry. However, based on interviews of 
the prosecutor's office in Norfolk, it was concluded that TRACER was under- 
utilized by the prosecutors. The court and police appeared to understand and 
use TRACER capabilities, but the prosecutor's office needed to be educated 
and upgraded especially in the use of management statistics. 

The above factor analysis showing no discernible impact occurred 
using variables that measure "all felonies" as one class. When classes of 
felonies were looked at separately, some changes were observed, as discussed 
in the performance section. The observed changes in pattern of felony 1 and 
2 pleas, average days from indictment to trial and court dismissal rates lent 
some support to a theory that TRACER was having some impact by helping to 
expedite case processing. A followup phone call to the Norfolk Circuit Court 
revealed that the court started to use TRACER outputs, during the observed 
time period, to examine the age of cases and as an aid in enforcing stricter 
controls on case processing. Emphasis was placed on the most serious charges 
in indictments by seeking quilty pleas in favor of dismissing lesser included 
charges. The court believed that as a direct result, the number of felony 1 
and 2 pleas increased, the rate of dismissals increased, and the average days 
from indictment to trial decreased. The data appeared to support the court's 
belief that TRACER had such an impact. The project staff also learned that a 
new chief judge took over about the time that the change was observed. He 
had a reputation for being tough and instituted a master calendaring procedure 
to preclude judge shopping, i.e., a judge was not assigned to a case until 
the day of trial. This change might also help to explain the observed change 
in disposition rates. 

A more conclusive time series analysis would be possible through 
the use of cross-sectional data involving a group of jurisdictions. This type^ 
of design would enable the inclusion of additional control variables. 
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D. Conclusions Regarding Feasibility Issues 

1 . Cost-Benefit Analysis 

Although several cost-benefit models exist and these models have 
been used for predictions (predominantly PROMIS sites), no evidence was , 

found among the 19 jurisdictions visited during this entire study that retro- 
spective cost-benefit analyses have been performed after PMIS implementation. 
Analysis after PMIS implementation will require close cooperation among the 
PMIS users, particularly to obtain agreement that users are willing to provide 
value judgments (i.e., give dollar values) for intangible benefits. 

The goals that a prosecutor establishes for his PMIS are helpful 
in identifying PMIS benefits. During this study, prosecutors were asked to 
identify their PMIS goals, and to rate the PMIS regarding "expected" and 
"actual" contributions toward goal attainments. The ratings were given on a 
scale of 1-100, but the same approach can be helpful in assigning a dollar 
value to related benefits. 

Many PMIS projects are implemented in phases. The most common 
practice is to start by entering data for one type of case (e.g., felonies). 
Once satisfied with the data entry and basic day-to-day operations of the 
system for the initial case type, the next type of case (e.g., misdemeanors) 
will be initiated. PMIS applications may also be implemented in phases. The 
trend is for daily outputs, such as calendars and schedules, to be initiated 
first, followed sometime later (usually months later) by management reports, 
such as statistical outputs. PMIS support to daily prosecution operations 
will reflect some benefits. If cost-benefit analysis is to be used in con- 
junction with the analysis of PMIS impact on the prosecution process, such an 
analysis should not be attempted until after management outputs have been 
produced for 9-12 months (which will also allow time for a "learning curve" 
regarding use of such outputs). 

Performing retrospective PMIS cost-benefit analysis is considered 
feasible provided that the PMIS is operational for 24-26 months prior to 
analysis. 



2. Availability of Baseline Data for A Priori Impact Analysis 

A feasible approach toward construction of the baseline is to com- 
bine the use of several data sources, i.e., available statistical reports and 
special tabulations drawn from a sampling of case records, to generate per- 
formance measures. 

Where reports on caseload and dispositions are available, they can 
provide a point of departure in constructing the baseline. Comparability and 
reliability of annual reports are open to question, and some effort needs to 
be made to check validity through an external source. A statistical sample 
of several hundred cases should be drawn manually from the files of cases 
terminated during the baseline year. Most data could be obtained from court 
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files. Some data could be obtained from the prosecutor's files Experience 
in the field investigations indicated that it would be feasible o construct 
such a data base for computation of the required performance measures if 
staff time of about 20 minutes per case could be allocated to the data collec- 
tion effort. 

3. Availability of Time Series Data for A Posteriori Impact Analysis 

During the field investigations, the manually maintained prosecution 
and court case files were examined to ascertain the feasibility of extracting 
data for constructing a time series. Most jurisdictions maintain their case 
files in sequence by court case number, and generally these numbers are 
assigned chronologically on the date the case is filed in court, i.e., accepted 
for prosecution. In order to obtain statistics on cases for which prosecution 
is declined, one would have to search the prosecutor's files. Such cases are 
usually filed by defendant's name or police complaint number. Thus, the most 
convenient method of selecting the sample is based on the date of case filing. 
If a large enough sample is drawn, starting with cases filed about one year 
before installation of the computer (to allow time for building up case termi- 
nations), then a representative time series could be constructed. The sample 
would need to be large enough to generate a sufficient sampling of all the 
desired subpopulations, e.g., cases declined, cases going to trial, cases 
dismissed, and other categories for which separate performance measures are 
desired. The size of the sample required will also depend on the purpose of 
the task. For example, to produce a set of baseline statistics for a single 
time frame, a sample of about 400 cases should be sufficient. However, to 
generate the complete time series data base for impact evaluation would 
require about 24 monthly summaries. To construct such a data base would 
probably require a sample exceeding 1,200 cases to be followed through to 
disposition. Using an estimated 20 minutes per file, about 400 person hours 
would be required to tabulate the data. The latter type of effort is con- 
sidered neither feasible nor necessary by the project staff. Since the 
purpose of the task is to evaluate the impact of the PMIS, it is reasonable 
to assume that the PMIS would be available to monitor its own performance, 
and require manual samplings only as supplemental data. 

The feasibility of collecting time series data from computer based 
records was investigated by obtaining computer tapes from Norfolk, Golden and 
Oakland.’ The Oakland tape was not readable on Westat's IBM 370 due to some 
hardware incompatibilities. Oklahoma City could have provided the tape, but 
required that a written request be submitted to the Governor's Commission for 
approval. This was not feasible, in view of some potential political sensi- 
tivities to release of such data and because of time constraints of the 
project. The Norfolk (IBM) and Golden (Honeywell) tapes were readable. The 
project budget did not permit the use of the Golden tapes in time series 
analysis. However, based on use of the Norfolk tapes, it was. the judgment of 
the staff that it would be feasible to construct a time series data base from 
the Golden tapes if more time and resources were made available. 
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4. Analytical Techniques 

Analytical techniques were examined for feasibility in three areas: 

• . cost analysis; 

• performance measurement; and 

• impact evaluation. 

In respect to the first two areas, there are major problems in con- 
structing an adequate data base for subsequent analysis. The need to define 
appropriate and comparable measures of cost and performance, and collect the 
required data over the relevant time periods are discussed elsewhere in this 
report. However, given uniformly defined data elements, the analytical tech- 
niques for estimating development and operational costs and productivity/ 
performance for prior periods of activity are conventional. Quantitative 
techniques for predicting project costs or criminal justice system performance 
are more challenging. Some predictive cost models were applied by sites, 
e.g., Portland's cost estimates are discussed elsewhere in this report. 

Various models for predicting criminal justice system performance exist in 
the field.* These are considered outside the scope of this feasibility 
study. 

The '"easibility issue examined, with respect to analytical tech- 
niques, concerned the adequacy of methods of evaluating the actual impact of 
the PMIS on the court environment. In accordance with the interrupted time 
series design presented earlier, a data base of monthly activity, spanning at 
least two years of criminal justice operations, was desired preferably 
including a period preceding the date the PMIS was considered operational. 
Given such a data base, techniques were needed to (1) detect any changes in 
the patterns of criminal justice system activity, and (2) determine, the 
extent to which those changes are attributable to the intervention of the 
PMIS. 



The data base assembled for the Norfolk TRACER system was used in 
examining the feasibility of applying interrupted time series analysis tech- 
niques for impact evaluation. The methods employed combined judgmental 
assessments of information obtained through on-site interviews and observa- 
tions, with information obtained through interpretations of statistical 
analyses of the time series data. 

Techniques for detecting changes in the patterns of criminal justice 
activity appeared to be sufficient. We examined monthly statistics on basic 
performance measures such as case load, rates of case terminations by plea, 
trial and dismissal, delay, fugitives apprehended and added,, and scheduling 

*For example, see "Criminal Justice Model: An Overview," J. Chalken, et al, 

National Institute of Law Enforcement and Criminal Justice, April 1976. 



activity. Bivariate relationships were analyzed first, i.e. , the data vari- 
ables were plotted against time to detect any changes in patterns. For 
example, was there a change in the rate of dismissals or in the average 
number of days from arrest to trial of felony cases? We next tried to adjust 
for the possibility of "noise" in these bivariate relationships, that is, the 
presence of errors, random variation, and confounding effects of one variable, 
on another. Factor analysis techniques were used and the factors were plotted 
against time. 

This analysis also involved hypothesis testing, i.e., examination 
of the results in the light of certain hypothesized effects of a PMIS, as 
discussed above in section III C. 3 /'Analysis of Time Series Data." The results 
of the statistical analysis were also compared with the judgmental assessments 
obtained from the site visit. The project staff concluded that while limited 
in scope, the techniques for detecting changes in patterns of criminal justice 
activity are reasonably sensitive and adequate. Again, a crucial aspect of 
this task is data base development. For example, data on comparative staff 
attitudes and capacities were obtained only through imprecise anecdotal and 
observational methods in one site visit. Such information obtained through 
more systematic observation over the time frame of analysis would have more 
accurately detected change. 

The second set of analytical techniques — to attribute changes to 
the PMIS intervention — is much more demanding. To attempt to perform this 
attribution, a non-experimental interrupted time series design was adopted as 
described earlier. A set of hypotheses was formulated to predict the poten- 
tial impact of the PMIS. Given such a theoretical framework with a suffici- 
ently rich set of control variables, supplemented by judgmental assessments 
derived from site interviews and observations, it was felt that a credible 
impact evaluation could be accomplished. The Norfolk PMIS was the only one 
for which a usable time series data base for impact measures could be 
constructed in the feasibility study. The data base was limited; it was 
sufficient for measuring change but not for attribution analysis due to an 
inadequate number of control variables. In the judgment of the project 
staff, an adequate time series data base could be constructed in the test 
sites, using the available PMIS to monitor changes over time, if sufficient 
time and resources were available. Given an adequate data base, the techni- 
ques, for analyzing time series data to test hypotheses are considered adequate 
for impact evaluation, especially if supplemented by judgmental assessments 
of experienced evaluation researchers. 
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IV. SUMMARY OF FINDINGS 



Introduction 



Issues emerging from this study have been discussed throughout the 
report. These discussions include findings that are based on analysis of the 
collected data, a literature search, opinions of PMIS users and planners, and 
judgmental assessments made by the study team. 

The purpose of this chapter is to summarize the findings of major impor- 
tance. References to the sections of this report where discussions of the 
findings can be found are shown parenthetically. 



B. Nature and Scope of PMIS Use 

Automated PMIS projects, are being used or planned primarily by larger 
(25 or more employees) prosecution offices, (Section II D.1). More PROMIS 
projects (as a group) are in use or being planned than any other single 
project, (Sections I B and II D.1). 

Types of PMIS projects in use include: systems dedicated to and con- 

trolled by the prosecutor; systems controlled by one agency (prosecutor or 
court, for example) and shared by the prosecutor and courts; integrated 
systems, controlled by a central data processing facility and shared by two 
or more user agencies; and, integrated systems, controlled by a central data 
processing facility and shared by multiple jurisdictions, as well as multiple 
agencies. PROMIS, as well as non-PROMIS projects, are employed in all of 
these types of systems, (Section IB). 

The most common and most effective type of PMIS is the integrated system. 
Advantages of the integrated system include: avoiding duplication of hard- 

ware, staffing and work effort; better data quality; and, greater depth in 
data processing expertise. Greater interagency cooperation and a higher 
degree of user satisfaction are found more frequently in integrated systems 
than in dedicated projects; and, confidentiality safeguards in the observed 
integrated systems were satisfactory (Sections II E.1 and II E.3.c). 

C. The State-of-the-Art in PMIS Use 

Assessments made by the project staff, during the first segment on-site 
surveys, indicated that many PMiS's were used effectively to support day-to- 
day case processing operations, but few were used effectively as strategic 
management tools, i.e., to support management policy development and decision- 
making. (For five of seven features supporting day-to-day operations, more 
than 90 percent of the PMIS projects had some capability and more than 00 
percent had state-of-the-art capability. The-same was true for only one of 
nine management features (Sections II E.3.b and II E 3.c). Nothing was 
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observed in the field investigations, during the second study segment, to 
change this impression. Indeed, the evidence gathered reinforced the con- 
tention that these systems are not yet being utilized to anywhere near their 
capacity as management information systems. In order for such systems to 
have an impact, they first must be understood and used. In the course of the 
site visits, the project staff found that many, prosecution managers did not 
sufficiently understand the potential of the PMIS as a tool for management. 

There are notable exceptions, however, to the above observations on the 
apathy of prosecution managers toward the PMIS as a management tool. Many 
district attorneys participating in the PROMIS Users Group are highly moti- 
vated toward understanding and using PROMIS as a tool, in policy development, 
management decisionmaking and court reform. The difference between PROMIS 
and non-PROMIS jurisdictions with respect to top management backing is quite 
noticeable. Non-PROMIS jurisdictions could benefit by establishing user 
organizations of their own, under the auspices of the National District 
Attorneys' Association or similar associations, for the purpose of sharing 
PMIS concepts and technology. 



D. PMIS Transfer Potential 

PROMIS is the most commonly transferred PMIS, with 37 jurisdictions 
already operating some version of the system on various hardware configura- 
tions, and 134 other jurisdictions reported as either in the transfer process 
or planning for PROMIS implementation (Section IB). 

The transfer of PROMIS was not proven by this study to be less costly 
than developing a PMIS from scratch. The need to modify PROMIS software to 
meet requirements of the local jurisdictions has been a primary factor contrib- 
uting to transfer costs (Sections II E 2.b and II E 4.b). 

The transfer of non-PROMIS projects is not a common practice and where 
transfer has taken place extensive software modifications have been necessary 
as with PROMIS (Section I B) . 

Detailed PMIS documentation is essential for technology transfer, but 
only five non-PROMIS projects surveyed had documentation that was considered 
adequate enough to support technology transfer (Section II E 2.e). Among the. 
well documented systems, however, interesting applications (e.g., automatic 
generation of arrest reports and warrants from Norfolk's .on-line booking 
operation - Section I B, and generation of charging documents by the Fort 
Worth PMIS - Section II E 3. a), and innovative approaches to programming 
(e.g., San Bernardino's interface control programs - Section II E 2.b) do 
offer the potential for transfer of non-PROMIS projects, or at least certain 
applications within those projects. 

PROMIS, on the other hand, has been well documented and the latest 
versions of PROMIS contain self-documenting features (Section II E.2.e). 

Several versions of PROMIS have been designed as a transferable system with 
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new capabilities and features being added at each stage of development. The 
MINI and MAXI PROMIS tailoring features and their capability to run on 
various hardware configurations offer prosecutors flexibility in adapting the 
system to meet. local reguirements and in selecting the type of computer system 
for their project (Section IB). 

/ 

Tailoring and self-documenting features of the MINI and MAXI PROMIS 
versions (Section I B) should help reduce transfer costs; however, experience 
with PROMIS tailoring had not progressed far enough at the time of the study 
to make this determination (Section II E 4.c). 



E. Funding of PMIS Projects 

PMIS projects, both PROMIS and non-PROMIS, have relied heavily on Federal 
funds and outside consultants to develop or transfer their systems (Section 
II E 2.c). Lack of money and people is the main reason, cited by both larger 
m and smaller prosecutor's offices, for not using or planning a PMIS (Section 

HE 2.d). 

Federal funding policies (e.g., LEAA's Incentive Fund Program) encourage 
PMIS transfer (Section II E 2.b), as indicated by the number of opera- 
tional, planned, and in-transfer PROMIS projects discussed above. Because of 
the extensive funding of PROMIS, LEAA asked that recommendations be developed 
as to where future PMIS funds should be concentrated (e.g., further PROMIS 
development, technical assistance for PROMIS transfers, or development of non- 
PROMIS projects). Based on the findings summarized above, it is recommended 
that future Federal funds be allocated in the following manner: 

f • First priority - technical assistance to PROMIS transfer projects , 

particularly where experienced data processing personnel are not 
readily available to cope with PROMIS implementation. In any case, 
consultants or contractors who have had experience in transferring 
PROMIS to one jurisdiction will be able to provide valuable 
assistance to other jurisdictions. If maximum benefits are to be 
gained from the heavy investment in PROMIS, effective implementa- 
tion of the systems should be the primary concern to LEAA. 

* Second priority - development of non-PROMIS projects . Although the 
latest PROMIS versions offer prosecutors flexibility in system 
design, there may be valid reasons for selecting a non-PROMIS 
| approach. For example, it may be more economical or more desirable 

to expand an existing criminal justice information system (e.g., a 
court or police system) to incorporate the prosecutor's require- 
•. ' ments, rather than attempting to interface PROMIS with the existing 

system or rather than implementing a separate system just for the 
prosecutor. Effective PMIS projects have been developed in this 
| manner, such as the TRACER system in Norfolk, VA, which started as 

a police system. 
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System modifications will be necessary in every transfer situation. 
PROMIS tailoring features may help simplify such modifications, but 
many data processors believe that the disadvantages of transfer 
(Section II E 2.b) outweigh the advantages; and, many of them hold 
to the premise that a PMIS should be designed to fit the specific 
procedures of the jurisdiction and that the jurisdiction should not 
have to alter procedures to fit a PMIS.* Efficiency of operations 
is another factor that may prompt modifications. In Golden, Colorado, 
for example, consideration was being given to modifying new PROMIS 
to utilize the central computer's own operating system rather than 
the one contained in the PROMIS package. 

Although most non-PROMIS systems may not be documented well enough 
to support system transfer (Section II E 2.e), innovative approaches, 
such as interface control programs developed in San Bernardino 
(Section II E 2.b), do offer the potential for transfer. PROMIS, 
therefore, is not the only alternative to PMIS support, and prosecu- 
tors should not be discouraged from examining those alternatives by 
denying them Federal funds for that purpose. The Justice Department 
should encourage innovation and non-PROMIS projects have demonstrated 
techniques that compare favorably with PROMIS projects. 

• Third priority - further PROMIS development . PROMIS has been in 
various stages of development since 1971. Each version of PROMIS, 
emerging from this developmental effort, has provided improvements 
over previous versions, with the latest one (MAXI PROMIS) offering 
the greatest potential for transfer because of increased flexibil- 
ities. Some jurisdictions have delayed PROMIS implementation 
pending the availability of the latest version.** Other jurisdic- 
tions, operating some version of PROMIS, are planning to change to 
MAXI PROMIS (Section II E 4.c). Among jurisdictions planning to 
change to MAXI PROMIS and among those planning a PMIS, but who have 
not decided on what system to acquire, PMIS planners are anxious to 
see the latest version in operation in order to assess its capabil- 
ities and to determine the degree of difficulty in implementing 
MAXI PROMIS.** Users and developers of automated projects are 
seldom completely satisfied with the operations of their system; 
they will (and should) continuously make improvements. At some 
point, however, priorities should be shifted from system development 
to system operations in order to gain maximum benefit from the 
developmental work already accomplished. 

Assigning priorities in this manner would have the effect of first 
capturing a return on Federal investments already made in repro- 
gramming and enhancing PROMIS for transfer; secondly, capturing 



* Based on interviews during on-site surveys. 

**Based on comment received from PMIS planners during Westat telephone survey 
of jurisdictions operating, transferring or. planning PROMIS. 
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benefits on investments made by local jurisdictions in criminal 
justice systems already in operation; and thirdly, allowing prose- 
cutors to gain more experience using existing applications before . 
investing resources in new enhancements. 

/ 

F. . Availability of Evaluation Data 

In general, data needed to evaluate PMIS projects are available; and, it 
is feasible to collect the data in varying degrees of detail and from various 
sources (Section III B). 

• Caseload data needed for time series analysis is available, but the 

source for those data will vary. Sources include: PMIS files 

(e.g., history tapes); monthly statistical reports (either PMIS or 
manually produced); and/or, case files from which data can be manual 
extracted (Section III B.1). 

• Data concerning total PMIS developmental and operational costs can 
be obtained, but will vary in the detailed backup data leading to 
those totals (Section III B.2). 

• PMIS benefits are relatively easy to identify, but many benefits are 
difficult to quantify because they require value judgments by the 
PMIS users (Sections III B.1 and III C.1). 



G* Feasibility of PMIS Evaluations 

1 . Cost-benefit Analysis 

Criminal justice agencies, other than the prosecutor, benefit from 
PMIS operations. All agencies with direct access to the system, or which 
receive PMIS outputs indirectly, should be included in any PMIS cost-benefit 
analysis (Section III C.1). 




I Cost-benefit analysis should be conducted only after the PMIS has 

been in operation long enough (24-26 months) to stabilize (i.e., planned 
dual operations, PMIS and manual functions, have been discontinued; all 
phases of PMIS implementation have been completed; and, users have reached 
the apex of the learning curve) (Section III D.1). 

I If a jurisdiction has made cost and benefit predictions (pre-PMIS 

implementation), the cost-benefit model used for those predictions should 
also be used when conducting the cost-benefit analysis (post-PMIS implementa- 
tion) (Section III B.1). 

Prior to conducting cost-benefit analysis, PMIS users within the 
& jurisdiction must agree that they are willing to provide dollar values for 

intangible benefits (Section III D.1). 





Under the conditions summarized above, performing retrospective 
PMIS cost-benefit analysis is considered feasible (Section III D.2). 

2. Performance Measures 

A number of performance measures were tested (e.g., numbers and 
rates of terminated cases, resulting from various actions; mean days from 
arrest through intermediate court actions to sentencing, and various workload 
measures) (Section III C.2). 

It was determined that a baseline or criminal justice performance 
for a priori evaluation can be constructed from: data contained in manually 

prepared statistical reports, where such reports are available; or, by 
manually abstracting data from case files, if sufficient resources (money and 
people) are made available (Section III B.1). 

It was further determined that data needed for a posteriori evalua- 
tions are available from PMIS files (Section III B.1): in some cases, peri- 

odic (monthly, for example) computations of performance measures are generated 
as part of scheduled PMIS processing (e.g., Norfolk and Oklahoma); in other 
cases, (e.g., Boston and Oakland) additional processing of available data can 
be programmed to generate the performance measures. 

3. Impact Evaluation 

The evaluation of PMIS impact on the prosecution/court, process 
requires the detection of changes in prosecution/court performance over time 
and a determination of whether such changes can be attributed to the PMIS 
(Section III C.3) . 

An interrupted time series analysis was tested as a method of 
detecting changes in prosecution/court performance. It was determined that 
available statistical techniques are adequate for detecting changes in the 
patterns of prosecution/court performance. It was also determined that these 
techniques are useful in determining whether changes can be attributed to the 
PMIS, but the PMIS data base must be supplemented by a rich set of control 
variables derived from on-site interviews and observations by experienced 
evaluation researchers (Section III D.4). 

H. PMIS Institutionalization || 

The extent of current PMIS use and planning for PMIS implementation 
(Sections I.B and II D.1) indicate a trend toward PMIS institutionalization. 

PMIS projects have demonstrated usefulness in on-line case and defendant 

status monitoring. Progress in utilizing the PMIS as a management tool has 

been very slow. The extent of future PMIS use will depend o.n a number of jj 

factors: the availability of external funds and technical assistance to f 

support PMIS development or transfer; the availability of funds and qualified 

people to operate and maintain the PMIS after external funds and technical 

assistance have been discontinued;, and the degree that PMIS operations meet 

prosecutor's expectations. 
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Since Federal Funding and outside technical assistance have been so 
essential to PMIS implementation thus far (Sections II E.2.c and II E.2.d), 

PMIS implementations can be expected to continue in the future only if such 
support is continued. 

In jurisdictions where the PMIS is operated by a centralized data pro- , 
cessing facility, retaining qualified personnel has been a problem because the 
pay scale for data processors is usually lower in' the local government than in 
the private sector.* In jurisdictions where the PMIS is operated by the prose- 
cutor, personnel in the prosecutor's office have been trained to operate the 
PMIS (as noted in three out of four prosecutor operated systems surveyed); 
these personnel are also looking forward to transferring their newly acquired 
skills to better paying jobs.* Between the two situations, the centralized 
facility offers the best opportunity for the prosecutor to obtain access to 
and retain qualified data processing personnel to support PMIS operations, 
because of the large personnel staff (Section I B) usually associated with 
such a facility. 

Local funds are currently used to sustain PMIS operations in those juris- 
dictions where Federal funding has expired. It is only reasonable, however, 
to expect local budgetary decisionmakers to insist on some evidence that PMIS 
benefits and/or impacts on the judicial system are worth their cost before 
increasingly scarce funds are allocated to continue PMIS operations. None of 
the jurisdictions surveyed had performed a retrospective cost-benefit analysis 
based on actual experience. However, cost-benefit predictions had been mace 
in some cases (Section III B.3). Although PMIS benefits have not been quanti- 
fied, top prosecution managers remain enthusiastic about their PMIS capabil- 
ities (Section III C.1), particularly where they have been directly involved 
in PMIS development and display a "pride in ownership" (Section II E.2.b). 

In these cases, user enthusiasm may offset the lack of documented evidence 
of PMIS worth, to justify funds for PMIS operations. In other cases, where 
prosecutor's expectations are not being met by the PMIS (Section II E.3.a), 
less support for continued PMIS operations can be expected. Therefore, 
in-depth PMIS evaluations need to be performed to provide users with suffi- 
cient justification for continued funding of their PMIS. 

In the opinion of the study team, PMIS utilization will continue to 
expand in the future provided that Federal funding is available to stimulate 
such projects. Unless the value of PMIS projects can be sufficiently demon- 
strated through independent evaluations, the availability of Federal funding 
can be expected to decline. It is also expected that .state and local govern- 
ment funding of PMIS projects will not pick up all the slack, unless cose 
effectiveness can be demonstrated through credible evaluation processes. 



♦Based on interviews with data processing personnel during site visits. 
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Los Angeles (Los Angeles County), California 
Neil Riddle, Data Systems Coordinator 

Robert Johnson, Assistant Director, Bureau of Special Operations 

Florence Linn, Assistant Director, Central Operations 

Michael Genelin, Head, Career Criminal Unit 

Joseph Siler, Special Assistant to District Attorney 

Larry Donoghue, Deputy District Attorney 

Eloise Williams, Data Systems Analyst 



Oakland (Alameda County), California 

D. Lowell Jensen, District Attorney 
Don Ingraham, Deputy District Attorney 
Rod Rolefson, Deputy District Attorney 
Kathy Bergland, Deputy District Attorney 
Richard Hsugner, Deputy District Attorney 
Ruby Freitas, Administrative Assistant 
Dave Budde, Administrative Assistant 
Peggy Richmon, Records Clerk 
Ninfa Wood, Secretary to District Attorney 
Don Whyte, Deputy District Attorney 
William M. Baldwin, Deputy District Attorney 
Bill McGuinness, Deputy District Attorney 
Yvonne Ayres, DALITE Manager/Programmer 
Diane Bullock, DALITE Operator 
William Kleeman, Deputy District Attorney 

Mike Scanlon, Administrative Office, Public Defender's Office 
Bill Cook, Records Section Director, Hayward Police Department 
C.J. Moret, Chief, Criminal Division, Clerk's Office, Oakland 
Piedmont Municipal Court 

Peggy Hunter, CORPUS Input Section, Alameda County Superior Court 
Dan George, CORPUS Project Manager 

Herbert L. Pike, Office of Court Administrator, Superior Court 
Beverly Graves, Criminal Clerk's Office 



Santa Ana (Orange County), California 



Truman T. Legg, Senior Systems Analyst 
Alan Slater, Assistant Court Administrator 
Walter F. Germond, Deputy District Attorney 
William J. Morrison, Administrative Services Officer, 

District Attorney's Office 

Don McClure, Manager, Systems and Programming, Computer Sciences 
Corporation 

Keith L. Concannon, Director, Orange County Criminal Justice Council 
Ross F. Penne, Center Director, Computer Sciences Corporation 



C-2 




James M. Cramer, District Attorney 

Rex Victor, Assistant District Attorney 

Kay Skawienski, Office of the Public Defender 

Debra A. Haskins, Data Processing Coordinator, San Bernardino County 
Clerk 

Jesse Pointer, Data Processing Coordinator, San Bernardino Municipal 
Courts 

Thomas H. Hudson, Manager, Automated Court Information System 

James R. Johnson, Vice President, Application Development Systems, Inc. 

San Jose (Santa Clara County), California 

Robert Webb, Assistant Prosecutor 
Jim Hagen, System Manager 

Ray Rule, Senior Management Analyst, County Executive Office 
Joel Berger, Adult Probation 
Alice Wheatiy, Municipal Court 
Pete Kiefer, Superior Court 



Golden, Colorado 

Nolan L. Brown, District Attorney 

Dan B. Fahrney, Chief Deputy District Attorney, District Courts 

Maurice H. Bennett, Jr., Administrator 

Kristen M. Beauchamp, Data Entry Technician 

Mary L. Simon, Data Entry Technician 

Larry Webster, PROMIS Supervisor 

Jerry L. Jorgenson, Chief Deputy District Attorney, Intake and Screening 
Ray Kechter, Chief Screening Officer 
Steve Siegal, Director, Pretrial Services 

C. Stephen- Cantrell, Deputy District Attorney, Preliminary Hearings 
Judi Webb, Paralegal, Preliminary Hearings 
Pat Blackard, Clerk, Preliminary Hearings 

Diane Edes, Systems Manager, Regional PROMIS, Colorado District 
Attorney's Council 

Deyrol E. Anderson, De'puty Director, Colorado District Attorney's Council 
Roger H. Allott, Chairman, Regional PROMIS Board of Directors and Chief 
Deputy District Attorney, 18th Judicial Circuit 
James Opp, Director, Jefferson County Department of Data Processing 
Don Haakinson, PROMIS Team Leader, Jefferson County Department of Data 
Processing 

Lt. Bruce Glasscock, Detective Division, Lakewood Police Department 
Honorable Anthony F. Vollock, Judge, First Judicial Circuit 
Honorable Daniel J. Shannon, Presiding Judge, First Judicial Circuit 
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District of Columbia 

Terry Russell, Special Assistant 

David Hetzel, Chief, Misdemeanors 

H. Greene,, Chief, Superior Court Division 

doe Valder, Deputy Chief, Grand Jury Unit 

Oohn Hume, Chief, Felony Trials 

John DePaolis, Assistant U.S. Attorney- 

Gloria Dellavalle, Chief, Systems 

John Middleton, Systems, D.C. Office of Cd Analysis 

Sue Ellen Hais, Systems, U.S. Attorney's Office 



Miami (Dade County), Florida 

Henry N. Adorno, Prosecution Management 
day Kolosky, Assistant Prosecutor 
Steven Levenson, Administrative Assistant 

Ed Peabody, Office of Computer Services and Information Systems 
Robert Castille, Systems Analyst, Dade County Criminal dustice Council 
Bill Stoiloff, Clerk's Office, Dade County Court 

Waukeeqan (Lake County), Illinois 

Randall Murphy, Administrator, Lake County Department of Management 
Services 

Richard Hilton, Department of Management Services 
Rhonda Brandhorst, Department of Management Services 
dohn Roberts, Project Leader - dustice Systems, Department of Manage- 
ment Services 

P. Randall Knowles, Assistant State's Attorney 
Honorable Harry D. Strouse, dudge, 19th dudicial Court 
Lt. Eugene McGaughey, Lake County Sheriff's Department 



Indianapolis (Marion County), Indiana 

Stephen Goldsmith, Marion County Prosecuting Attorney 

Beth Walpole, PROMIS Coordinator, Office of the Prosecuting Attorney 

Bill Divine, Deputy Prosecuting Attorney 

Bill O'Connor, Marion County Data Processing 

E.W. (Chick) Wieting, Business Manager, Office of the Prosecuting 
Attorney 
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Louisville (d e fferson County), Kentucky 



Paul Richwelvky, First Assistant 
Richard Cooper, Assistant Prosecutor 
William Chiquelin, CATCH Project Manager 



/ 



New Orleans (Orleans Parish), Louisiana 

Ralph Capatelli, First Deputy 

dohn Meyer, Assistant Prosecutor 

Robert Early, Assistant Prosecutor 

Lance Afrik, Assistant Prosecutor 

Denis Waldron, Assistant Prosecutor 

Emmett Fremaux, Chief Deputy Clerk, District Court 

Glen Christina, System Manager 

dim Rousselle, Assistant System Manager 




Barbara Daly, Office of the State's Attorney 
Maryann Willin, Deputy State's Attorney 

Mike Nieberding, Project Director, State dudicial Information Systems 
dim Salb, Project Manage, Baltimore Courts Project 
George Riggin, Criminal Assignment Commissioner, Supreme Bench of 
Baltimore 

Linda Crowley, State dudicial Information Systems 



\ 



i Boston (Suffolk County), Massachusetts 

Dave Rodman, Executive Assistant to District Attorney 
Paul Buckley, First Assistant to District Attorney 
dim Caffrey, Assistant Prosecutor 
Bob Long, Assistant Prosecutor 

' George Gushue, Office Supervisor, Assignments Section 

dohn Duffett, Systems Manager, Assignments Section 
Mary McCarthy, Data Recorder/Coder 
Bernie Dwyer, Assistant Prosecutor 
Bob Powers, Assistant Prosecutor 
dim Lynch, Assistant Prosecutor 
) Daniel C. Mullane, Assistant Prosecutor 

Marion Walsh, Legal Administrative Assistant 
Detective dohn V. Nee, Boston Metropolitan Police Department 
Bob Mitchell, dudicial Information System, Superior Court, 
Middlesex County 

Bob Stacey, dudicial Information System, Superior Court, 

) Middlesex County 
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Oklahoma City, Oklahoma 



Dr. Glen Wallace, Director, Statistical Analysis Center (SAC), 

Oklahoma Crime Commission 
dim Wilson, ADRS System Manager, SAC 
don Steen-, Data Analyst, SAC 
Neal Gilson, ADRS Training Officer, SAC 
d. Patrick Sweeney, Systems Analyst, SAC 

Del Woodruff, Oklahoma County District’ Attorney's Office, System Manager 
dane Bluejacket, Programmer 

Paul D. Boyd, Chief, Identification Section, Oklahoma State Bureau of 
Investigation 

Tom Elliott, Director, District Attorney's Training Coordination Council 
Niles dackson, Attorney, District Attorney's Training Coordination 
Council 

Tom Thompson, First Assistant District Attorney, Pottawatomi County, 
Shawnee, OK 



Portland (Multnomah County), Oregon 



dack Pessia, PROMIS Coordinator 
Chrys A. Martin, Staff Assistant 
Suzanne Lewis, Data Input Control Clerk 
Dorthea Anderson 
Kelly Bacon, Executive Assistant 
Wayne C. Pearson 

dack Wilson, Systems Analyst, Data Processing Authority 
Bob Davidson, Financial Administrator, Data Processing Authority 
Charles Benard, Criminal Coordinator, Circuit Court 
Adele Goggins, System Specialist, Circuit Court 



Fort Worth (Tarrant County), Texas 

Wayne Hyde, System Manager 

d. d. Heinemann,- Assistant Prosecutor 

Steve Chaney, Assistant Prosecutor 



Norfolk, Virginia 



Tommy Miller, Assistant Prosecutor 

Tom Baldwin, Administrative Assistant, Commonwealth's Attorney's Office 

d.W. Nixon, Data Processing Manager, General Services 

A.C. Hooper, Clerk of Court's Office, District Court 

Charlie Greene, Clerk of Court's Office, Circuit Court 

Bill Garbee, Systems Analyst, Data Processing Division 

Sgt. D.H. Mason, Central Files Div-ision, Norfolk Police Department 
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Capt. Niel Koch, Commander, Central Files' Division, Norfolk Police 
Department 

Martin Mendelsohn, Director of General Services 
Rich Nichols, Systems Analyst, Data Processing Division 
dim Barnhill, Norfolk Police Department 
Ginger Nicholson, Commonwealth Attorney's Office 
Carol Marx, Commonwealth Attorney's Office 
? Tom Rutherford, Commonwealth Attorney's Office 

; Lee Ann Diller, Systems Analyst, Data Processing Division 

% 

V ■ 

Milwaukee (Milwaukee County), Wisconsin 

I Louis A. Metz, III, dudicial Information Systems Coordinator, Clerk 

L of Courts 

1 Honorable William Gardner, dudge 

Sgt. Richard Krizan, Milwaukee County Sheriff's Office 
\ Donald Thorgaard, Chief Deputy, Clerk of Court's Criminal Division 

I Robert Erdman, Calendar Clerk, Clerk of Court's Criminal Division 

! Franklin Lotter, Superintendent, Milwaukee County House of Corrections 

Herman B. dohn, Deputy District Attorney 




